Skip to Main content Skip to Navigation
New interface
Journal articles

LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR

Florent Bartoccioni 1, 2 Éloi Zablocki 2 Patrick Pérez 2 Matthieu Cord 2 Karteek Alahari 1 
1 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
Abstract : Vision-based depth estimation is a key feature in autonomous systems, which often relies on a single camera or several independent ones. In such a monocular setup, dense depth is obtained with either additional input from one or several expensive LiDARs, e.g., with 64 beams, or camera-only methods, which suffer from scale-ambiguity and infinite-depth problems. In this paper, we propose a new alternative of densely estimating metric depth by combining a monocular camera with a light-weight LiDAR, e.g., with 4 beams, typical of today's automotive-grade mass-produced laser scanners. Inspired by recent self-supervised methods, we introduce a novel framework, called LiDARTouch, to estimate dense depth maps from monocular images with the help of ``touches'' of LiDAR, i.e., without the need for dense ground-truth depth. In our setup, the minimal LiDAR input contributes on three different levels: as an additional model's input, in a self-supervised LiDAR reconstruction objective function, and to estimate changes of pose (a key component of self-supervised depth estimation architectures). Our LiDARTouch framework achieves new state of the art in self-supervised depth estimation on the KITTI dataset, thus supporting our choices of integrating the very sparse LiDAR signal with other visual features. Moreover, we show that the use of a few-beam LiDAR alleviates scale ambiguity and infinite-depth issues that camera-only methods suffer from. We also demonstrate that methods from the fully-supervised depth-completion literature can be adapted to a self-supervised regime with a minimal LiDAR signal.
Document type :
Journal articles
Complete list of metadata
Contributor : THOTH Team Connect in order to contact the contributor
Submitted on : Monday, November 28, 2022 - 2:51:24 PM
Last modification on : Thursday, December 1, 2022 - 3:29:31 AM


Files produced by the author(s)


  • HAL Id : hal-03508099, version 2
  • ARXIV : 2109.03569




Florent Bartoccioni, Éloi Zablocki, Patrick Pérez, Matthieu Cord, Karteek Alahari. LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR. Computer Vision and Image Understanding, In press. ⟨hal-03508099v2⟩



Record views


Files downloads