Camera and LiDAR analysis for 3D object detection in foggy weather conditions

Analyse par caméra et LiDAR pour la détection d'objets 3D par temps de brouillard

Nguyen Anh Minh Mai (1, 2) , Pierre Duthon (2) , Pascal Housam Salmane (2) , Louahdi Khoudour (2) , Alain Crouzil (1) , Sergio A. Velastin (3, 4)

1 IRIT-MINDS - CoMputational imagINg anD viSion
2 STI - Systèmes de transports intelligents
3 EECS - School of Electronic Engineering and Computer Science
4 Carlos III University of Madrid

Today, the popularity of self-driving cars is growing at an exponential rate and is starting to creep onto the roads of developing countries. For autonomous vehicles to function, one of the essential features that needs to be developed is the ability to perceive their surroundings. To do this, sensors such as cameras, LiDAR, or radar are integrated to collect raw data. The objective of this paper is to evaluate a fusion solution of cameras and LiDARs (4 and 64 beams) for 3D object detection in foggy weather conditions. The data from the two input sensors are fused and an analysis of the contribution of each sensor on its own is then performed. In our analysis, we calculate average precision using the popular KITTI dataset, on which we have applied different intensities of fog (on a dataset we have called Multifog KITTI). The main results observed are as follows. Performances with stereo camera and 4 or 64 beams LiDAR are high (90.15%, 89.26%). Performance of the 4 beams LiDAR alone decreases sharply in foggy weather conditions (13.43%). Performance when using only a camera-based model remains quite high (89.36%). In conclusion, stereo cameras on their own are capable of detecting 3D objects in foggy weather with high accuracy and their performance slightly improves when used in conjunction with LIDAR sensors.

Mots clés: self-driving car; 3D object detection; foggy weather condition; sensor fusion; deep learning

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Nguyen Anh Minh Mai, Pierre Duthon, Pascal Housam Salmane, Louahdi Khoudour, Alain Crouzil, et al.. Camera and LiDAR analysis for 3D object detection in foggy weather conditions. 12th International Conference on Pattern Recognition Systems (ICPRS 2022), MINES Saint-Etienne (France); Chilean Association for Pattern Recognition (ACHiRP, a member of the IAPR); IEEE France SP Chapter; IEEE Chile CIS Chapter, Jun 2022, Saint-Etienne, France. pp.1-7, ⟨10.1109/ICPRS54038.2022.9854073⟩. ⟨hal-03820137⟩