A Patch-Based Algorithm for Diverse and High Fidelity Single Image Generation

Nicolas Cherel (1, 2) , Andrés Almansa (3) , Yann Gousseau (2, 1) , Alasdair Newson (1, 2)

1 LTCI - Laboratoire Traitement et Communication de l'Information
2 IMAGES - Image, Modélisation, Analyse, GEométrie, Synthèse
3 MAP5 - UMR 8145 - Mathématiques Appliquées Paris 5

Image generation is the task of producing new samples from one or several example images. Until recently, this has been done using large image databases, in particular using Generative Adversarial Networks (GANs). However, Shaham et al. [1] recently proposed the SinGAN method, which achieves this generation using a single image example. At the same time, researchers are realizing that classical patchbased methods can replace certain neural networks, with no costly training. In this paper, we present a purely patch-based method, named Patches for Single image generation (PSin), which requires no training and generates samples in seconds. Our algorithm is based on the minimization of a global, patchbased energy functional, which ensures the visual fidelity of the result to the original image. We also ensure diversity of the results by carefully choosing the initialization of the algorithm. We propose two initialization variants. We compare our results to both the original SinGAN and another recent patch-based image generation approach, both qualitatively and quantitatively using multiple metrics.

Keywords: patch; single image generation; generative adversarial networks

Domains: Image Processing [eess.IV]

Nicolas Cherel, Andrés Almansa, Yann Gousseau, Alasdair Newson. A Patch-Based Algorithm for Diverse and High Fidelity Single Image Generation. 29th IEEE International Conference on Image Processing (ICIP) 2022, Oct 2022, Bordeaux, France. ⟨hal-03822204⟩