Combining Depth, Color and Position Information for Object Instance Recognition on an Indoor Mobile Robot

Louis-Charles Caron 1, 2
2 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : Mobile robots have already entered people's homes to perform simple tasks for them. For robots at home to become real assistants, they have to be able to recognize the objects in their owner's home. In this thesis, object instance recognition algorithms are designed to cope with the variations (viewing angle, lighting conditions, etc.) that occur in the context of mobile robotics experiments. Several ways to take advantage of this context are studied. First, a geometric segmentation algorithm that benefits from the structure of indoor scenes to find isolated objects is designed. Then, a neural network based object recognition process that fuses shape, color and texture information provided by color and depth cameras is presented. This step highlights the importance of carefully combining multiple features and the difficulty to use color information in robotics. Finally, an alternative approach using a nearest neighbor classifier, which is easier to train than the neural network, is detailed. It relies on physical measures available to the robot to eliminate some parameters in clustering and nearest neighbor search procedures. Also, it uses information gathered through multiple sightings of the objects to reduce the negative impact of occlusions. The algorithm can recognize 52 objects with a success rate of 80\% and runs in 500 ms on average (on an Intel Core i5 CPU with 3~GB of RAM) on a mobile robot. This work shows that identifying and taking advantage of the structure and information available is essential to handle the variations that happen in indoor mobile robot experiments.
Complete list of metadatas
Contributor : Louis-Charles Caron <>
Submitted on : Wednesday, January 6, 2016 - 11:58:53 AM
Last modification on : Wednesday, July 3, 2019 - 10:48:05 AM
Long-term archiving on : Thursday, April 7, 2016 - 4:17:53 PM



  • HAL Id : tel-01251481, version 1


Louis-Charles Caron. Combining Depth, Color and Position Information for Object Instance Recognition on an Indoor Mobile Robot. Computer Vision and Pattern Recognition [cs.CV]. ENSTA ParisTech, 2015. English. ⟨tel-01251481⟩



Record views


Files downloads