The skyline as a marker for augmented reality in urban context

Mehdi Ayadi (1, 2) , Leo Valque , Mihaela Scuturici (2) , Serge Miguet (2) , Chokri Ben Amar (1)

1 REGIM-Lab - REsearch Group in Intelligent Machines [Sfax]
2 imagine - Extraction de Caractéristiques et Identification

In recent years, augmented reality (AR) technologies have emerged as powerful tools to help visualize the future impacts of new constructions on cities. Many approaches that use costly sensors and height-end platforms to run AR in real-time have been developed. Little efforts have been made to embed AR on mobile phones. In this paper, we present a novel approach that uses the Skyline as a marker in an AR system. This lightweight feature enables real-time matching of virtual and real skyline on smartphones. We use device's embedded instruments to estimate the user's pose. This approximation is used to insert a synthetic object in the live video stream. This first approach gives a very unrealistic impression of the viewed scene: the inserted objects appear to hover and float with the user's movements. In order to address this problem, we use the live video camera feed as additional source of information which provides a redundancy to the instruments estimation. We extract the Skyline (a set of pixels that defines the boundary between the building and the sky) as main visual feature. Our proposal is to use these automatically extracted points and track them throughout the video sequence, to allow synthetic objects to anchor these visual features, making it possible to simulate a a landscape from multiple viewpoints using a smartphone. We use images of the Lyon city (France) to illustrate our proposal.

Mots clés: Skyline; Augmented reality; 3D city model

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement des images [eess.IV]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Mehdi Ayadi, Leo Valque, Mihaela Scuturici, Serge Miguet, Chokri Ben Amar. The skyline as a marker for augmented reality in urban context. 13th International Symposium on Visual Computing, Nov 2018, Las Vegas, United States. ⟨hal-01872631⟩