3D PixMix: Image-Inpainting in 3D Environments

Shohei Mori, Jan Herling, Wolfgang Broll, Norihiko Kawai, Hideo Saito, Dieter Schmalstieg, and Denis Kalkofen. 3d pixmix: image-inpainting in 3d environments. In Adjunct Proceedings of the IEEE International Symposium for Mixed and Augmented Reality 2018 (To appear). 2018.
[BibTeX▼]

@inproceedings{ismar2018_pcs_poster_1026,
    author = "Mori, Shohei and Herling, Jan and Broll, Wolfgang and Kawai, Norihiko and Saito, Hideo and Schmalstieg, Dieter and Kalkofen, Denis",
    title = "3D PixMix: Image-Inpainting in 3D Environments",
    year = "2018",
    booktitle = "Adjunct Proceedings of the IEEE International Symposium for Mixed and Augmented Reality 2018 (To appear)"
}

Abstract

State of the art methods for diminished reality (DR) propagate pixel information from keyframes to later frames to achieve real-time image inpainting in 3D spaces. However, this approach assumes a planar scene and produces artifacts, if the scene geometry is not sufficiently planar. In this paper, we present 3D PixMix, a new real-time inpainting method that addresses non-planar scenes by considering both color and depth information in the inpainting process. We define cost functions for both the color and the geometric appearance in the inpainting scheme and use an RGB-D sensor for depth fusion using a SLAM. Comparison results against the conventional PixMix show that 3D PixMix obtains the equivalent or even better quality in 3D scenes with additional depth information.

Sponsors

Abstract