HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Journal articles

Mini-batch learning of exponential family finite mixture models

Hien D Nguyen 1 Florence Forbes 2, 3 Geoffrey Mclachlan 4
2 MISTIS - Modelling and Inference of Complex and Structured Stochastic Systems
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
Abstract : Mini-batch algorithms have become increasingly popular due to the requirement for solving optimization problems, based on large-scale data sets. Using an existing online expectation-maximization (EM) algorithm framework, we demonstrate how mini-batch (MB) algorithms may be constructed, and propose a scheme for the stochastic stabilization of the constructed mini-batch algorithms. Theoretical results regarding the convergence of the mini-batch EM algorithms are presented. We then demonstrate how the mini-batch framework may be applied to conduct maximum likelihood (ML) estimation of mixtures of exponential family distributions, with emphasis on ML estimation for mixtures of normal distributions. Via a simulation study, we demonstrate that the mini-batch algorithm for mixtures of normal distributions can outperform the standard EM algorithm. Further evidence of the performance of the mini-batch framework is provided via an application to the famous MNIST data set.
Document type :
Journal articles
Complete list of metadata

Cited literature [53 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02415068
Contributor : Florence Forbes Connect in order to contact the contributor
Submitted on : Thursday, March 26, 2020 - 2:35:19 PM
Last modification on : Friday, February 4, 2022 - 3:34:27 AM
Long-term archiving on: : Saturday, June 27, 2020 - 2:31:22 PM

File

20190612_R1-FF2.pdf
Files produced by the author(s)

Identifiers

Citation

Hien D Nguyen, Florence Forbes, Geoffrey Mclachlan. Mini-batch learning of exponential family finite mixture models. Statistics and Computing, Springer Verlag (Germany), 2020, 30, pp.731-748. ⟨10.1007/s11222-019-09919-4⟩. ⟨hal-02415068v2⟩

Share

Metrics

Record views

71

Files downloads

93