Convolutional neural network architecture for geometric matching

Ignacio Rocco (1, 2) , Relja Arandjelovic (1, 2) , Josef Sivic (1, 2, 3)

1 WILLOW - Models of visual object recognition and scene understanding
2 PSL - Université Paris Sciences et Lettres
3 CTU - Czech Technical University in Prague

We address the problem of determining correspondences between two images in agreement with a geometric model such as an affine or thin-plate spline transformation, and estimating its parameters. The contributions of this work are threefold. First, we propose a convolutional neural network architecture for geometric matching. The architecture is based on three main components that mimic the standard steps of feature extraction, matching and simultaneous in-lier detection and model parameter estimation, while being trainable end-to-end. Second, we demonstrate that the network parameters can be trained from synthetically generated imagery without the need for manual annotation and that our matching layer significantly increases generalization capabilities to never seen before images. Finally, we show that the same model can perform both instance-level and category-level matching giving state-of-the-art results on the challenging Proposal Flow dataset.

Domaines: Informatique [cs]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Apprentissage [cs.LG]

Ignacio Rocco, Relja Arandjelovic, Josef Sivic. Convolutional neural network architecture for geometric matching. CVPR 2017 - IEEE Conference on Computer Vision and Pattern Recognition , Jul 2017, Honolulu, United States. ⟨hal-01513001⟩