FeGAN: Scaling Distributed GANs - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

FeGAN: Scaling Distributed GANs

Résumé

Existing approaches to distribute Generative Adversarial Networks (GANs) either (i) fail to scale for they typically put the two components of a GAN (the generator and the discriminator) on different machines, inducing significant communication overhead, or (ii) they face GAN training specific issues, exacerbated by distribution. We propose FeGAN, the first middleware for distributing GANs over hundreds of devices addressing the issues of mode collapse and vanishing gradients. Essentially, we revisit the idea of Federated Learning, co-locating a generator with a discriminator on each device (addressing the scaling problem) and having a server aggregate the devices' models using balanced sampling and Kullback-Leibler (KL) weighting, mitigating training issues and boosting convergence. Through extensive experiments, we show that FeGAN generates high-quality dataset samples in a scalable and devices' heterogeneity tolerant manner. In particular, FeGAN achieves up to 5× throughput gain with 1.5× less bandwidth compared to the state-of-the-art GAN distributed approach (named MD-GAN), while scaling to at least one order of magnitude more devices. We demonstrate that FeGAN boosts training by 2.6× w.r.t. a baseline application of Federated Learning to GANs, while preventing training issues.
Fichier principal
Vignette du fichier
middleware2020-GGKLM-FeGAN-preprint.pdf (1.21 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03118260 , version 1 (22-01-2021)

Identifiants

Citer

Rachid Guerraoui, Arsany Guirguis, Anne-Marie Kermarrec, Erwan Le Merrer. FeGAN: Scaling Distributed GANs. ACM/IFIP Middleware 2020 - Annual ACM/IFIP Middleware conference, Dec 2020, Delft / Virtual, Netherlands. ⟨10.1145/3423211.3425688⟩. ⟨hal-03118260⟩
448 Consultations
888 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More