Stochastic algorithms with descent guarantees for ICA

Pierre Ablin 1 Alexandre Gramfort 1 Jean-François Cardoso 2 Francis Bach 3, 4
1 PARIETAL - Modelling brain structure, function and variability based on high-field MRI data
NEUROSPIN - Service NEUROSPIN, Inria Saclay - Ile de France
4 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : Independent component analysis (ICA) is a widespread data exploration technique, where observed signals are modeled as linear mixtures of independent components. From a machine learning point of view, it amounts to a matrix factorization problem with a statistical independence criterion. Infomax is one of the most used ICA algorithms. It is based on a loss function which is a non-convex log-likelihood. We develop a new majorization-minimization framework adapted to this loss function. We derive an online algorithm for the streaming setting, and an incremental algorithm for the finite sum setting, with the following benefits. First, unlike most algorithms found in the literature, the proposed methods do not rely on any critical hyper-parameter like a step size, nor do they require a line-search technique. Second, the algorithm for the finite sum setting, although stochas-tic, guarantees a decrease of the loss function at each iteration. Experiments demonstrate progress on the state-of-the-art for large scale datasets, without the necessity for any manual parameter tuning.
Complete list of metadatas

Cited literature [38 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02372092
Contributor : Pierre Ablin <>
Submitted on : Wednesday, November 20, 2019 - 11:24:41 AM
Last modification on : Monday, February 10, 2020 - 6:13:44 PM

File

ablin19a.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02372092, version 1

Citation

Pierre Ablin, Alexandre Gramfort, Jean-François Cardoso, Francis Bach. Stochastic algorithms with descent guarantees for ICA. AISTATS 2019 - 22nd International Conference on Artificial Intelligence and Statistics, Apr 2019, Naha, Japan. ⟨hal-02372092⟩

Share

Metrics

Record views

20

Files downloads

21