Deep Relighting Networks for Image Light Source Manipulation

Li-Wen Wang (1) , Wan-Chi Siu (1) , Zhi-Song Liu (2) , Chu-Tak Li (1) , Daniel P K Lun (1)

1 POLYU - The Hong Kong Polytechnic University [Hong Kong]
2 LIX - Laboratoire d'informatique de l'École polytechnique [Palaiseau]

Manipulating the light source of given images is an interesting task and useful in various applications, including photography and cinematography. Existing methods usually require additional information like the geometric structure of the scene, which may not be available for most images. In this paper, we formulate the single image relighting task and propose a novel Deep Relighting Network (DRN) with three parts: 1) scene reconversion , which aims to reveal the primary scene structure through a deep auto-encoder network, 2) shadow prior estimation, to predict light effect from the new light direction through adversarial learning, and 3) re-renderer, to combine the primary structure with the reconstructed shadow view to form the required estimation under the target light source. Experiments show that the proposed method outperforms other possible methods, both qualitatively and quantitatively. Specifically, the proposed DRN has achieved the best PSNR in the "AIM2020-Any to one relighting challenge" of the 2020 ECCV conference.

Mots clés: Image relighting; back-projection theory; deep learn- ing

Domaines: Informatique [cs]

Li-Wen Wang, Wan-Chi Siu, Zhi-Song Liu, Chu-Tak Li, Daniel P K Lun. Deep Relighting Networks for Image Light Source Manipulation. The 2020 European Conference on Computer Vision, Sep 2020, Online, United Kingdom. ⟨hal-03014846⟩