A Neural Tangent Kernel Perspective of GANs

Jean-Yves Franceschi (1) , Emmanuel de Bézenac (1) , Ibrahim Ayed (1, 2) , Mickaël Chen (3) , Sylvain Lamprier (1) , Patrick Gallinari (1, 4)

1 MLIA - Machine Learning and Information Access
2 ThereSIS lab - Thales
3 Valeo.ai
4 Criteo AI Lab

Theoretical analyses for Generative Adversarial Networks (GANs) generally assume an arbitrarily large family of discriminators and do not consider the characteristics of the architectures used in practice. We show that this framework of analysis is too simplistic to properly analyze GAN training. To tackle this issue, we leverage the theory of infinite-width neural networks to model neural discriminator training for a wide range of adversarial losses via its Neural Tangent Kernel (NTK). Our analytical results show that GAN trainability primarily depends on the discriminator's architecture. We further study the discriminator for specific architectures and losses, and highlight properties providing a new understanding of GAN training. For example, we find that GANs trained with the integral probability metric loss minimize the maximum mean discrepancy with the NTK as kernel. Our conclusions demonstrate the analysis opportunities provided by the proposed framework, which paves the way for better and more principled GAN models. We release a generic GAN analysis toolkit based on our framework that supports the empirical part of our study.

Domaines: Machine Learning [stat.ML]; Apprentissage [cs.LG]; Réseau de neurones [cs.NE]

Jean-Yves Franceschi, Emmanuel de Bézenac, Ibrahim Ayed, Mickaël Chen, Sylvain Lamprier, et al.. A Neural Tangent Kernel Perspective of GANs. 2021. ⟨hal-03254591v1⟩