Adversarial training of partially invertible variational autoencoders

Thomas Lucas (1) , Konstantin Shmelkov (1) , Karteek Alahari (1) , Cordelia Schmid (1) , Jakob Verbeek (1)

1 Thoth - Apprentissage de modèles à partir de données massives

Adversarial generative image models yield outstanding sample quality, but suffer from two drawbacks: (i) they mode-drop, i.e., do not cover the full support of the target distribution, and (ii) they do not allow for likelihood evaluations on held-out data. Conversely, maximum likelihood estimation encourages models to cover the full support of the training data, but yields poor samples. To address these mutual shortcomings, we propose a generative model that can be jointly trained with both procedures. In our approach, the conditional independence assumption typically made in variational autoencoders is relaxed by leveraging invertible models. This leads to improved sample quality, as well as improved likelihood on heldout data. Our model significantly improves on existing hybrid models, yielding GAN-like samples, and IS and FID scores that are competitive with fully adversarial models, while offering likelihoods measures on held-out data comparable to recent likelihood-based methods.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Thomas Lucas, Konstantin Shmelkov, Karteek Alahari, Cordelia Schmid, Jakob Verbeek. Adversarial training of partially invertible variational autoencoders. INNF'19 - Workshop on Invertible Neural Nets and Normalizing Flows, Jun 2019, Long Beach, United States. pp.1-14. ⟨hal-01886285v2⟩