Skip to Main content Skip to Navigation
Conference papers

Repurposing a Relighting Network for Realistic Compositions of Captured Scenes

Abstract : Multi-view stereo can be used to rapidly create realistic virtual content, such as textured meshes or a geometric proxy for free-viewpoint Image-Based Rendering (IBR). These solutions greatly simplify the content creation process compared to traditional methods, but it is difficult to modify the content of the scene. We propose a novel approach to create scenes by composing (parts of) multiple captured scenes. The main difficulty of such compositions is that lighting conditions in each captured scene are different; to obtain a realistic composition we need to make lighting coherent. We propose a two-pass solution, by adapting a multi-view relighting network. We first match the lighting conditions of each scene separately and then synthesize shadows between scenes in a subsequent pass. We also improve the realism of the composition by estimating the change in ambient occlusion in contact areas between parts and compensate for the color balance of the different cameras used for capture. We illustrate our method with results on multiple compositions of outdoor scenes and show its application to multi-view image composition, IBR and textured mesh creation.
Complete list of metadatas

Cited literature [36 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02500771
Contributor : Baptiste Nicolet <>
Submitted on : Friday, March 6, 2020 - 2:00:12 PM
Last modification on : Tuesday, May 26, 2020 - 6:50:59 PM
Document(s) archivé(s) le : Sunday, June 7, 2020 - 4:19:21 PM

Identifiers

Citation

Baptiste Nicolet, Julien Philip, George Drettakis. Repurposing a Relighting Network for Realistic Compositions of Captured Scenes. Symposium on Interactive 3D Graphics and Games (I3D ’20), May 2020, San Francisco, United States. ⟨10.1145/3384382.3384523⟩. ⟨hal-02500771⟩

Share

Metrics

Record views

240

Files downloads

259