Skip to Main content Skip to Navigation
Journal articles

X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data

Abstract : This paper addresses the problem of semi-supervised transfer learning with limited cross-modality data in remote sensing. A large amount of multi-modal earth observation images, such as multispectral imagery (MSI) or synthetic aperture radar (SAR) data, are openly available on a global scale, enabling parsing global urban scenes through remote sensing imagery. However, their ability in identifying materials (pixel-wise classification) remains limited, due to the noisy collection environment and poor discriminative information as well as limited number of well-annotated training images. To this end, we propose a novel cross-modal deep-learning framework, called X-ModalNet, with three well-designed modules: self-adversarial module, interactive learning module, and label propagation module, by learning to transfer more discriminative information from a small-scale hyperspectral image (HSI) into the classification task using a large-scale MSI or SAR data. Significantly, X-ModalNet generalizes well, owing to propagating labels on an updatable graph constructed by high-level features on the top of the network, yielding semi-supervised cross-modality learning. We evaluate X-ModalNet on two multi-modal remote sensing datasets (HSI-MSI and HSI-SAR) and achieve a significant improvement in comparison with several state-of-the-art methods.
Document type :
Journal articles
Complete list of metadata
Contributor : Vincent Couturier-Doux Connect in order to contact the contributor
Submitted on : Monday, February 15, 2021 - 6:53:33 PM
Last modification on : Friday, January 7, 2022 - 3:44:36 AM

Links full text



Danfeng Hong, Naoto Yokoya, Gui-Song Xia, Jocelyn Chanussot, Xiao Xiang Zhu. X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data. ISPRS Journal of Photogrammetry and Remote Sensing, Elsevier, 2020, 167, pp.12-23. ⟨10.1016/j.isprsjprs.2020.06.014⟩. ⟨hal-03142183⟩



Les métriques sont temporairement indisponibles