Parallelized Stochastic Gradient Markov Chain Monte Carlo Algorithms for Non-Negative Matrix Factorization

Umut Şimşekli (1, 2) , Alain Durmus (1, 2) , Roland Badeau (1, 2) , Gael Richard (1, 2) , Éric Moulines (3) , Taylan Cemgil (4)

1 LTCI - Laboratoire Traitement et Communication de l'Information
2 Télécom ParisTech
3 CMAP - Centre de Mathématiques Appliquées - Ecole Polytechnique
4 Department of Computer Engineering [Bogazici]

Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non- negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

Mots clés: Stochastic Gradient MCMC; Non-Negative Matrix Factorization; Tweedie Distribution; Beta Divergence; Richardson-Romberg Extrapolation.

Domaines: Traitement du signal et de l'image [eess.SP]; Machine Learning [stat.ML]

Umut Şimşekli, Alain Durmus, Roland Badeau, Gael Richard, Éric Moulines, et al.. Parallelized Stochastic Gradient Markov Chain Monte Carlo Algorithms for Non-Negative Matrix Factorization. 42nd International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, Mar 2017, New Orleans, United States. ⟨hal-01416357⟩