Efficient Posterior Sampling For Diverse Super-Resolution with Hierarchical VAE Prior

Jean Prost (1) , Antoine Houdard (2) , Andrés Almansa (3) , Nicolas Papadakis (1, 4)

1 IMB - Institut de Mathématiques de Bordeaux
2 Ubisoft
3 MAP5 - UMR 8145 - Mathématiques Appliquées Paris 5
4 MONC - Modélisation Mathématique pour l'Oncologie

We investigate the problem of producing diverse solutions to an image super-resolution problem. From a probabilistic perspective, this can be done by sampling from the posterior distribution of an inverse problem, which requires the definition of a prior distribution on the high-resolution images. In this work, we propose to use a pretrained hierarchical variational autoencoder (HVAE) as a prior. We train a lightweight stochastic encoder to encode low-resolution images in the latent space of a pretrained HVAE. At inference, we combine the low-resolution encoder and the pretrained generative model to super-resolve an image. We demonstrate on the task of face super-resolution that our method provides an advantageous trade-off between the computational efficiency of conditional normalizing flows techniques and the sample quality of diffusion based methods.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Jean Prost, Antoine Houdard, Andrés Almansa, Nicolas Papadakis. Efficient Posterior Sampling For Diverse Super-Resolution with Hierarchical VAE Prior. VISAPP 2024 - 19th International Conference on Computer Vision Theory and Applications, Feb 2024, Rome, Italy. ⟨10.5220/0012352800003660⟩. ⟨hal-03675314⟩