Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

Maximilian Jaritz; Raoul de Charette; Emilie Wirbel; Xavier Perrotton; Fawzi Nashashibi

Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

Maximilian Jaritz ^{1, 2} Raoul de Charette ¹ Emilie Wirbel ² Xavier Perrotton ² Fawzi Nashashibi ¹

1 RITS - Robotics & Intelligent Transportation Systems

Inria de Paris

2 VALEO PES Company - Valeo Equipements Electriques Moteur

Abstract : Convolutional neural networks are designed for dense data, but vision data is often sparse (stereo depth, point clouds, pen stroke, etc.). We present a method to handle sparse depth data with optional dense RGB, and accomplish depth completion and semantic segmentation changing only the last layer. Our proposal efficiently learns sparse features without the need of an additional validity mask. We show how to ensure network robustness to varying input sparsities. Our method even works with densities as low as 0.8% (8 layer lidar), and outperforms all published state-of-the-art on the Kitti depth completion benchmark.

Document type :

Conference papers

Domain :

Computer Science [cs] / Computer Vision and Pattern Recognition [cs.CV]

Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01858241
Contributor : Maximilian Jaritz <>
Submitted on : Monday, August 20, 2018 - 11:54:38 AM
Last modification on : Tuesday, November 27, 2018 - 2:26:55 PM

Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

Links full text

Identifiers

Collections

Citation

Export

Metrics

112

Contact

Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

Links full text

Identifiers

Collections

Citation

Export

Share

Metrics

112

Contact