Skip to Main content Skip to Navigation
Journal articles

Algorithms for nonnegative matrix factorization with the beta-divergence

Abstract : This letter describes algorithms for nonnegative matrix factorization (NMF) with the β-divergence (β-NMF). The β-divergence is a family of cost functions parameterized by a single shape parameter β that takes the Euclidean distance, the Kullback-Leibler divergence, and the Itakura-Saito divergence as special cases (β = 2, 1, 0 respectively). The proposed algorithms are based on a surrogate auxiliary function (a local majorization of the criterion function). We first describe a majorization-minimization algorithm that leads to multiplicative updates, which differ from standard heuristic multiplicative updates by a β-dependent power exponent. The monotonicity of the heuristic algorithm can, however, be proven for β (0, 1) using the proposed auxiliary function. Then we introduce the concept of the majorization-equalization (ME) algorithm, which produces updates that move along constant level sets of the auxiliary function and lead to larger steps than MM. Simulations on synthetic and real data illustrate the faster convergence of the ME approach. The letter also describes how the proposed algorithms can be adapted to two common variants of NMF: penalized NMF (when a penalty function of the factors is added to the criterion function) and convex NMF (when the dictionary is assumed to belong to a known subspace).
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-00626110
Contributor : Jérôme Idier <>
Submitted on : Friday, September 23, 2011 - 2:52:22 PM
Last modification on : Thursday, January 7, 2021 - 8:18:14 PM

Identifiers

Citation

Cédric Févotte, Jérôme Idier. Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, Massachusetts Institute of Technology Press (MIT Press), 2011, 23 (9), pp.2421-2456. ⟨10.1162/NECO_a_00168⟩. ⟨hal-00626110⟩

Share

Metrics

Record views

337