Skip to Main content Skip to Navigation
Conference papers

Neural Network Fusion of Color, Depth and Location for Object Instance Recognition on a Mobile Robot

Louis-Charles Caron 1, 2 David Filliat 1, 2 Alexander Gepperth 2, 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : The development of mobile robots for domestic assistance re-quires solving problems integrating ideas from different fields of research like computer vision, robotic manipulation, localization and mapping. Semantic mapping, that is, the enrichment a map with high-level infor-mation like room and object identities, is an example of such a complex robotic task. Solving this task requires taking into account hard software and hardware constraints brought by the context of autonomous mobile robots, where short processing times and low energy consumption are mandatory. We present a light-weight scene segmentation and object in-stance recognition algorithm using an RGB-D camera and demonstrate it in a semantic mapping experiment. Our method uses a feed-forward neural network to fuse texture, color and depth information. Running at 3 Hz on a single laptop computer, our algorithm achieves a recognition rate of 97% in a controlled environment, and 87% in the adversarial con-ditions of a real robotic task. Our results demonstrate that state of the art recognition rates on a database does not guarantee performance in a real world experiment. We also show the benefit in these conditions of fusing several recognition decisions and data from different sources. The database we compiled for the purpose of this study is publicly available.
Document type :
Conference papers
Complete list of metadata

Cited literature [33 references]  Display  Hide  Download
Contributor : David Filliat <>
Submitted on : Wednesday, November 26, 2014 - 9:14:05 AM
Last modification on : Thursday, January 21, 2021 - 9:26:01 AM
Long-term archiving on: : Friday, February 27, 2015 - 10:57:19 AM


Files produced by the author(s)


  • HAL Id : hal-01087392, version 1



Louis-Charles Caron, David Filliat, Alexander Gepperth. Neural Network Fusion of Color, Depth and Location for Object Instance Recognition on a Mobile Robot. Second Workshop on Assistive Computer Vision and Robotics (ACVR), in conjunction with European Conference on Computer Vision, Sep 2014, Zurich, Switzerland. ⟨hal-01087392⟩



Record views


Files downloads