Shape Reconstruction Using Volume Sweeping and Learned Photoconsistency

Vincent Leroy 1 Jean-Sébastien Franco 1 Edmond Boyer 1
1 MORPHEO - Capture and Analysis of Shapes in Motion
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : The rise of virtual and augmented reality fuels an increased need for contents suitable to these new technologies including 3D contents obtained from real scenes. We consider in this paper the problem of 3D shape reconstruction from multi-view RGB images. We investigate the ability of learning-based strategies to effectively benefit the reconstruction of arbitrary shapes with improved precision and robustness. We especially target real life performance capture, containing complex surface details that are difficult to recover with existing approaches. A key step in the multi-view reconstruction pipeline lies in the search for matching features between viewpoints in order to infer depth information. We propose to cast the matching on a 3D receptive field along viewing lines and to learn a multi-view photoconsistency measure for that purpose. The intuition is that deep networks have the ability to learn local photometric configurations in a broad way, even with respect to different orientations along various viewing lines of the same surface point. Our results demonstrate this ability, showing that a CNN, trained on a standard static dataset, can help recover surface details on dynamic scenes that are not perceived by traditional 2D feature based methods. Our evaluation also shows that our solution compares on par to state of the art reconstruction pipelines on standard evaluation datasets, while yielding significantly better results and generalization with realistic performance capture data.
Type de document :
Communication dans un congrès
European Conference on Computer Vision, Sep 2018, Munich, Germany
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01849286
Contributeur : Vincent Leroy <>
Soumis le : vendredi 17 août 2018 - 11:43:06
Dernière modification le : jeudi 13 décembre 2018 - 15:32:17

Fichiers

Identifiants

  • HAL Id : hal-01849286, version 2

Collections

Citation

Vincent Leroy, Jean-Sébastien Franco, Edmond Boyer. Shape Reconstruction Using Volume Sweeping and Learned Photoconsistency. European Conference on Computer Vision, Sep 2018, Munich, Germany. 〈hal-01849286v2〉

Partager

Métriques

Consultations de la notice

238

Téléchargements de fichiers

237