PAVI: Plate-Amortized Variational Inference

Louis Rouillard (1) , Alexandre Le Bris (1) , Thomas Moreau (1) , Demian Wassermann (1)

1 MIND - Modèles et inférence pour les données de Neuroimagerie

Given observed data and a probabilistic generative model, Bayesian inference searches for the distribution of the model's parameters that could have yielded the data. Inference is challenging for large population studies where millions of measurements are performed over a cohort of hundreds of subjects, resulting in a massive parameter space. This large cardinality renders off-the-shelf Variational Inference (VI) computationally impractical. In this work, we design structured VI families that efficiently tackle large population studies. Our main idea is to share the parameterization and learning across the different i.i.d. variables in a generative model -symbolized by the model's . We name this concept . Contrary to off-the-shelf stochastic VI --which slows down inference-- plate amortization results in orders of magnitude faster to train variational distributions. Applied to large-scale hierarchical problems, PAVI yields expressive, parsimoniously parameterized VI with an affordable training time --effectively unlocking inference in those regimes. We illustrate the practical utility of PAVI through a challenging Neuroimaging example featuring 400 million latent parameters, demonstrating a significant step towards scalable and expressive Variational Inference.

Domaines: Machine Learning [stat.ML]; Intelligence artificielle [cs.AI]; Apprentissage [cs.LG]; Neurosciences [q-bio.NC]; Méthodologie [stat.ME]

Louis Rouillard, Alexandre Le Bris, Thomas Moreau, Demian Wassermann. PAVI: Plate-Amortized Variational Inference. Transactions on Machine Learning Research Journal, 2023. ⟨hal-03684389v2⟩