Proposal Flow

Bumsub Ham (1, 2) , Minsu Cho (1, 2) , Cordelia Schmid (3) , Jean Ponce (1, 4, 2)

1 LIENS - Laboratoire d'informatique de l'école normale supérieure
2 WILLOW - Models of visual object recognition and scene understanding
3 Thoth - Apprentissage de modèles à partir de données massives
4 PSL - Université Paris sciences et lettres

Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout, typical in scene flow computation. We introduce a novel approach to this problem, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing scene flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that proposal flow can effectively be transformed into a conventional dense flow field. We introduce a new dataset that can be used to evaluate both general scene flow techniques and region-based approaches such as proposal flow. We use this benchmark to compare different matching algorithms, object proposals, and region features within proposal flow with the state of the art in scene flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing scene flow methods in various settings.

Domaines: Informatique [cs]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Bumsub Ham, Minsu Cho, Cordelia Schmid, Jean Ponce. Proposal Flow. CVPR 2016 - IEEE Conference on Computer Vision & Pattern Recognition, Jun 2016, LAS VEGAS, United States. ⟨hal-01240281v2⟩