Skip to Main content Skip to Navigation
Book sections

Visual geo-localization of non-photographic depictions via 2D-3D alignment

Mathieu Aubry 1, 2, 3 Bryan C. Russell 4 Josef Sivic 5, 6
3 imagine [Marne-la-Vallée]
CSTB - Centre Scientifique et Technique du Bâtiment, ENPC - École des Ponts ParisTech, ligm - Laboratoire d'Informatique Gaspard-Monge
6 WILLOW - Models of visual object recognition and scene understanding
CNRS - Centre National de la Recherche Scientifique : UMR8548, Inria Paris-Rocquencourt, DI-ENS - Département d'informatique de l'École normale supérieure
Abstract : This chapter describes a technique that can geo-localize arbitrary 2D depictions of architectural sites, including drawings, paintings and historical pho-tographs. This is achieved by aligning the input depiction with a 3D model of the corresponding site. The task is very difficult as the appearance and scene structure in the 2D depictions can be very different from the appearance and geometry of the 3D model, e.g., due to the specific rendering style, drawing error, age, lighting or change of seasons. In addition, we face a hard search problem: the number of possible alignments of the depiction to a set of 3D models from different architec-tural sites is huge. To address these issues, we develop a compact representation of complex 3D scenes. 3D models of several scenes are represented by a set of discrim-inative visual elements that are automatically learnt from rendered views. Similar to object detection, the set of visual elements, as well as the weights of individual features for each element, are learnt in a discriminative fashion. We show that the learnt visual elements are reliably matched in 2D depictions of the scene despite large variations in rendering style (e.g. watercolor, sketch, historical photograph) and structural changes (e.g. missing scene parts, large occluders) of the scene. We demonstrate that the proposed approach can automatically identify the correct archi-tectural site as well as recover an approximate viewpoint of historical photographs and paintings with respect to the 3D model of the site. Fig. 1 Our system automatically geo-localizes paintings, drawings, and historical photographs by recovering their viewpoint with respect to a geo-referenced 3D model of the depicted architec-tural site. Here geo-localized paintings of Notre Dame in Paris are visualized in the Google Earth geobrowser.
Complete list of metadatas

Cited literature [50 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01119203
Contributor : Mathieu Aubry <>
Submitted on : Saturday, February 21, 2015 - 11:37:16 PM
Last modification on : Wednesday, February 26, 2020 - 7:06:12 PM
Document(s) archivé(s) le : Tuesday, May 26, 2015 - 10:21:15 AM

File

chapter02.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01119203, version 1

Citation

Mathieu Aubry, Bryan C. Russell, Josef Sivic. Visual geo-localization of non-photographic depictions via 2D-3D alignment. Springer. Visual Analysis and Geolocalization of Large-Scale Imagery, 2015. ⟨hal-01119203⟩

Share

Metrics

Record views

1290

Files downloads

765