Robust RGB-D Fusion for Saliency Detection - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Robust RGB-D Fusion for Saliency Detection

Résumé

Efficiently exploiting multi-modal inputs for accurate RGB-D saliency detection is a topic of high interest. Most existing works leverage cross-modal interactions to fuse the two streams of RGB-D for intermediate features' enhancement. In this process, a practical aspect of the low quality of the available depths has not been fully considered yet. In this work, we aim for RGB-D saliency detection that is robust to the low-quality depths which primarily appear in two forms: inaccuracy due to noise and the misalignment to RGB. To this end, we propose a robust RGB-D fusion method that benefits from (1) layer-wise, and (2) trident spatial, attention mechanisms. On the one hand, layerwise attention (LWA) learns the trade-off between early and late fusion of RGB and depth features, depending upon the depth accuracy. On the other hand, trident spatial attention (TSA) aggregates the features from a wider spatial context to address the depth misalignment problem. The proposed LWA and TSA mechanisms allow us to efficiently exploit the multi-modal inputs for saliency detection while being robust against low-quality depths. Our experiments on five benchmark datasets demonstrate that the proposed fusion method performs consistently better than the state-of-the-art fusion alternatives. The source code is publicly available at: https://github.com/Zongwei97/RFnet.
Fichier principal
Vignette du fichier
3DV2022_Robust_RGB_D_Fusion_for_Saliency_Detection.pdf (3.83 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03746242 , version 1 (05-08-2022)
hal-03746242 , version 2 (30-08-2022)

Identifiants

  • HAL Id : hal-03746242 , version 2

Citer

Zongwei Wu, Shriarulmozhivarman Gobichettipalayam, Brahim Tamadazte, Guillaume Allibert, Danda Pani Paudel, et al.. Robust RGB-D Fusion for Saliency Detection. 10th International Conference on 3D Vision, Sep 2022, Prague, Czech Republic. ⟨hal-03746242v2⟩
138 Consultations
74 Téléchargements

Partager

Gmail Facebook X LinkedIn More