Detecting Overfitting of Deep Generative Networks via Latent Recovery

Ryan Webster 1 Julien Rabin 1 Loïc Simon 1 Frédéric Jurie 1
1 Equipe Image - Laboratoire GREYC - UMR6072
GREYC - Groupe de Recherche en Informatique, Image, Automatique et Instrumentation de Caen
Abstract : State of the art deep generative networks are capable of producing images with such incredible realism that they can be suspected of memorizing training images. It is why it is not uncommon to include visualizations of training set nearest neighbors, to suggest generated images are not simply memorized. We demonstrate this is not sufficient and motivates the need to study memorization/overfitting of deep generators with more scrutiny. This paper addresses this question by i) showing how simple losses are highly effective at reconstructing images for deep generators ii) analyzing the statistics of reconstruction errors when reconstructing training and validation images, which is the standard way to analyze overfitting in machine learning. Using this methodology, this paper shows that overfitting is not detectable in the pure GAN models proposed in the literature, in contrast with those using hybrid adversarial losses, which are amongst the most widely applied generative methods. The paper also shows that standard GAN evaluation metrics fail to capture memorization for some deep generators. Finally, the paper also shows how off-the-shelf GAN generators can be successfully applied to face inpainting and face super-resolution using the proposed reconstruction method, without hybrid adversarial losses.
Complete list of metadatas
Contributor : Julien Rabin <>
Submitted on : Tuesday, May 14, 2019 - 10:36:37 AM
Last modification on : Wednesday, May 15, 2019 - 7:17:10 AM

Links full text


  • HAL Id : hal-02128249, version 1
  • ARXIV : 1901.03396


Ryan Webster, Julien Rabin, Loïc Simon, Frédéric Jurie. Detecting Overfitting of Deep Generative Networks via Latent Recovery. IEEE conference on computer vision and pattern recognition, Jun 2019, Long Beach, United States. ⟨hal-02128249⟩



Record views