Sampling from non-smooth distributions through Langevin diffusion

Tung Duy Luu (1) , Jalal M. Fadili (2) , Christophe Chesneau (3)

1 NU - Normandie Université
2 Equipe Image - Laboratoire GREYC - UMR6072
3 LMNO - Laboratoire de Mathématiques Nicolas Oresme

In this paper, we propose proximal splitting-type algorithms for sampling from distributions whose densities are not necessarily smooth nor log-concave. Our approach brings together tools from, on the one hand, variational analysis and non-smooth optimization, and on the other hand, stochastic diffusion equations, and in particular the Langevin diffusion. We establish in particular consistency guarantees of our algorithms seen as discretization schemes in this context. These algorithms are then applied to compute the exponentially weighted aggregates for regression problems involving non-smooth penalties that are commonly used to promote some notion of simplicity/complexity. Some popular penalties are detailed and implemented on some numerical experiments.

Domaines: Statistiques [math.ST]; Traitement du signal et de l'image [eess.SP]; Analyse numérique [math.NA]; Optimisation et contrôle [math.OC]; Théorie [stat.TH]; Machine Learning [stat.ML]

Tung Duy Luu, Jalal M. Fadili, Christophe Chesneau. Sampling from non-smooth distributions through Langevin diffusion. Methodology and Computing in Applied Probability, In press, 23, pp.1173-1201. ⟨10.1007/s11009-020-09809-7⟩. ⟨hal-02896662⟩