Invertible Flow Non Equilibrium sampling

Achille Thin (1) , Yazid Janati (2, 3, 4) , Sylvain Le Corff (2, 3, 4) , Charles Ollion (1) , Arnaud Doucet (5) , Alain Durmus (6) , Eric Moulines (1) , Christian Robert (7, 8)

1 CMAP - Centre de Mathématiques Appliquées - Ecole Polytechnique
2 IP Paris - Institut Polytechnique de Paris
3 TIPIC-SAMOVAR - Traitement de l'Information Pour Images et Communications
4 TSP - CITI - Communications, Images et Traitement de l'Information
5 Department of Statistics [Oxford]
6 CMLA - Centre de Mathématiques et de Leurs Applications
7 CEREMADE - CEntre de REcherches en MAthématiques de la DEcision
8 Department of Statistics [Warwick]

Simultaneously sampling from a complex distribution with intractable normalizing constant and approximating expectations under this distribution is a notoriously challenging problem. We introduce a novel scheme, Invertible Flow Non Equilibrium Sampling (InFine), which departs from classical Sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) approaches. InFine constructs unbiased estimators of expectations and in particular of normalizing constants by combining the orbits of a deterministic transform started from random initializations. When this transform is chosen as an appropriate integrator of a conformal Hamiltonian system, these orbits are optimization paths. InFine is also naturally suited to design new MCMC sampling schemes by selecting samples on the optimization paths. Additionally, InFine can be used to construct an Evidence Lower Bound (ELBO) leading to a new class of Variational AutoEncoders (VAE).

Domaines: Méthodologie [stat.ME]; Machine Learning [stat.ML]

Achille Thin, Yazid Janati, Sylvain Le Corff, Charles Ollion, Arnaud Doucet, et al.. Invertible Flow Non Equilibrium sampling. 2021. ⟨hal-03168489v1⟩