Parsing Facades with Shape Grammars and Reinforcement Learning

Abstract : In this paper, we use shape grammars (SGs) for facade parsing, which amounts to segmenting 2D building facades into balconies, walls, windows, and doors in an architecturally meaningful manner. The main thrust of our work is the introduction of reinforcement learning (RL) techniques to deal with the computational complexity of the problem. RL provides us with techniques such as Q-learning and state aggregation which we exploit to efficiently solve facade parsing. We initially phrase the 1D parsing problem in terms of a Markov Decision Process, paving the way for the application of RL-based tools. We then develop novel techniques for the 2D shape parsing problem that take into account the specificities of the facade parsing problem. Specifically, we use state aggregation to enforce the symmetry of facade floors and demonstrate how to use RL to exploit bottom-up, image-based guidance during optimization. We provide systematic results on the Paris building dataset and obtain state-of-the-art results in a fraction of the time required by previous methods. We validate our method under diverse imaging conditions and make our software and results available online.
Document type :
Journal articles
Liste complète des métadonnées
Contributor : Enzo Ferrante <>
Submitted on : Thursday, August 29, 2013 - 4:30:36 PM
Last modification on : Thursday, February 7, 2019 - 5:29:11 PM




Olivier Teboul, Iasonas Kokkinos, Loïc Simon, Koutsourakis Panagiotis, Nikos Paragios. Parsing Facades with Shape Grammars and Reinforcement Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2013, 35 (7), pp.1744-1756. ⟨10.1109/TPAMI.2012.252⟩. ⟨hal-00855609⟩



Record views