Real-time RGB-D semantic keyframe SLAM based on image segmentation learning from industrial CAD models

Abstract : This paper presents methods for performing real-time semantic SLAM aimed at autonomous navigation and control of a humanoid robot in a manufacturing scenario. A novel multi-keyframe approach is proposed that simultaneously minimizes a semantic cost based on class-level features in addition to common photometric and geometric costs. The approach is shown to robustly construct a 3D map with associated class labels relevant to robotic tasks. Alternatively to existing approaches, the segmentation of these semantic classes have been learnt using RGB-D sensor data aligned with an industrial CAD manufacturing model to obtain noisy pixel-wise labels. This dataset confronts the proposed approach in a complicated real-world setting and provides insight into the practical use case scenarios. The semantic segmentation network was fine tuned for the given use case and was trained in a semi-supervised manner using noisy labels. The developed software is real-time and integrated with ROS to obtain a complete semantic reconstruction for the control and navigation of the HRP4 robot. Experiments in-situ at the Airbus manufacturing site in Saint-Nazaire validate the proposed approach.
Complete list of metadatas

Cited literature [44 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02391499
Contributor : Andrew Comport <>
Submitted on : Tuesday, December 3, 2019 - 3:43:37 PM
Last modification on : Saturday, December 7, 2019 - 1:41:44 AM

File

ICAR19_0187_FI.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02391499, version 1

Collections

Citation

Howard Mahe, Denis Marraud, Andrew I. Comport. Real-time RGB-D semantic keyframe SLAM based on image segmentation learning from industrial CAD models. International Conference on Advanced Robotics, Dec 2019, Belo Horizonte, Brazil. ⟨hal-02391499⟩

Share

Metrics

Record views

45

Files downloads

65