Skip to Main content Skip to Navigation
Conference papers

Real-time RGB-D semantic keyframe SLAM based on image segmentation learning from industrial CAD models

Abstract : This paper presents methods for performing real-time semantic SLAM aimed at autonomous navigation and control of a humanoid robot in a manufacturing scenario. A novel multi-keyframe approach is proposed that simultaneously minimizes a semantic cost based on class-level features in addition to common photometric and geometric costs. The approach is shown to robustly construct a 3D map with associated class labels relevant to robotic tasks. Alternatively to existing approaches, the segmentation of these semantic classes have been learnt using RGB-D sensor data aligned with an industrial CAD manufacturing model to obtain noisy pixel-wise labels. This dataset confronts the proposed approach in a complicated real-world setting and provides insight into the practical use case scenarios. The semantic segmentation network was fine tuned for the given use case and was trained in a semi-supervised manner using noisy labels. The developed software is real-time and integrated with ROS to obtain a complete semantic reconstruction for the control and navigation of the HRP4 robot. Experiments in-situ at the Airbus manufacturing site in Saint-Nazaire validate the proposed approach.
Complete list of metadatas

Cited literature [44 references]  Display  Hide  Download
Contributor : Andrew Comport <>
Submitted on : Tuesday, December 3, 2019 - 3:43:37 PM
Last modification on : Monday, October 12, 2020 - 10:30:31 AM
Long-term archiving on: : Wednesday, March 4, 2020 - 8:07:31 PM


Files produced by the author(s)


  • HAL Id : hal-02391499, version 1



Howard Mahe, Denis Marraud, Andrew I. Comport. Real-time RGB-D semantic keyframe SLAM based on image segmentation learning from industrial CAD models. International Conference on Advanced Robotics, Dec 2019, Belo Horizonte, Brazil. ⟨hal-02391499⟩



Record views


Files downloads