Stochastic Image Models from SIFT-like descriptors

Agnès Desolneux (1) , Arthur Leclaire (1)

1 CMLA - Centre de Mathématiques et de Leurs Applications

Extraction of local features constitutes a first step of many algorithms used in computer vision. The choice of keypoints and local features is often driven by the optimization of a performance criterion on a given computer vision task, which sometimes makes the extracted content difficult to apprehend. In this paper we propose to examine the content of local image descriptors from a reconstruction perspective. For that, relying on the keypoints and descriptors provided by the scale-invariant feature transform (SIFT), we propose two stochastic models for exploring the set of images that can be obtained from given SIFT descriptors. The two models are both defined as solutions of generalized Poisson problems that combine gradient information at different scales. The first model consists in sampling an orientation field according to a maximum entropy distribution constrained by local histograms of gradient orientations (at scale 0). The second model consists in simple resampling of the local histogram of gradient orientations at multiple scales. We show that both these models admit convolutive expressions which allow to compute the model statistics (e.g. the mean, the variance). Also, in the experimental section, we show that these models are able to recover many image structures, while not requiring any external database. Finally, we compare several other choices of points of interest in terms of quality of reconstruction, which confirms the optimality of the SIFT keypoints over simpler alternatives.

Domaines: Traitement du signal et de l'image [eess.SP]; Probabilités [math.PR]

Agnès Desolneux, Arthur Leclaire. Stochastic Image Models from SIFT-like descriptors. SIAM Journal on Imaging Sciences, 2018, ⟨10.1137/18M116592X⟩. ⟨hal-01692139⟩