Generative Adversarial Network for Pansharpening with Spectral and Spatial Discriminators

Anaïs Gastineau (1) , Jean-François Aujol (1) , Yannick Berthoumieu (2) , Christian Germain (2)

1 IMB - Institut de Mathématiques de Bordeaux
2 IMS - Laboratoire de l'intégration, du matériau au système

The pansharpening problem amounts to fusing a high-resolution panchromatic image with a low-resolution multispectral image so as to obtain a high-resolution multispectral image. So the preservation of the spatial resolution of the panchromatic image and the spectral resolution of the multispectral image are of key importance for the pansharpening problem. To cope with it, we propose a new method based on a bi-discriminator in a Generative Adversarial Network (GAN) framework. The first discriminator is optimized to preserve textures of images by taking as input the luminance and the near infrared band of images, and the second discriminator preserves the color by comparing the chroma components Cb and Cr. Thus, this method allows to train two discriminators, each one with a different and complementary task. Moreover, to enhance these aspects, the proposed method based on bi-discriminator, and called MDSSC-GAN SAM, considers a spatial and a spectral constraints in the loss function of the generator. We show the advantages of this new method on experiments carried out on Pléiades and World View 3 satellite images.

Mots clés: pansharpening; remote sensing; multi-discriminator; Generative Adversarial Network; Deep learning

Domaines: Mathématiques [math]

Anaïs Gastineau, Jean-François Aujol, Yannick Berthoumieu, Christian Germain. Generative Adversarial Network for Pansharpening with Spectral and Spatial Discriminators. 2020. ⟨hal-02906109v2⟩