Learning Dense Correspondence via 3D-guided Cycle Consistency - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2018

Learning Dense Correspondence via 3D-guided Cycle Consistency

Tinghui Zhou
  • Fonction : Auteur
Philipp Krähenbühl
  • Fonction : Auteur
Mathieu Aubry
Qixing Huang
  • Fonction : Auteur

Résumé

Discriminative deep learning approaches have shown impressive results for problems where human-labeled ground truth is plentiful, but what about tasks where labels are difficult or impossible to obtain? This paper tackles one such problem: establishing dense visual correspondence across different object instances. For this task, although we do not know what the ground-truth is, we know it should be consistent across instances of that category. We exploit this consistency as a supervisory signal to train a convolutional neural network to predict cross-instance correspondences between pairs of images depicting objects of the same category. For each pair of training images we find an appropriate 3D CAD model and render two synthetic views to link in with the pair, establishing a correspondence flow 4-cycle. We use ground-truth synthetic-to-synthetic correspondences, provided by the rendering engine, to train a ConvNet to predict synthetic-to-real, real-to-real and real-to-synthetic correspondences that are cycle-consistent with the ground-truth. At test time, no CAD models are required. We demonstrate that our end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods in correspondence-related tasks.

Dates et versions

hal-01801042 , version 1 (28-05-2018)

Identifiants

Citer

Tinghui Zhou, Philipp Krähenbühl, Mathieu Aubry, Qixing Huang, Alexei A. Efros. Learning Dense Correspondence via 3D-guided Cycle Consistency. 2018. ⟨hal-01801042⟩
94 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More