Subsampled online matrix factorization with convergence guarantees

Arthur Mensch 1 Julien Mairal 2 Gaël Varoquaux 1 Bertrand Thirion 3, 1
1 PARIETAL - Modelling brain structure, function and variability based on high-field MRI data
Inria Saclay - Ile de France
2 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
3 NEUROSPIN - Service NEUROSPIN
Abstract : We present a matrix factorization algorithm that scales to input matrices that are large in both dimensions (i.e., that contains more than 1TB of data). The algorithm streams the matrix columns while subsampling them, resulting in low complexity per iteration and reasonable memory footprint. In contrast to previous online matrix factorization methods, our approach relies on low-dimensional statistics from past iterates to control the extra variance introduced by subsampling. We present a convergence analysis that guarantees us to reach a stationary point of the problem. Large speed-ups can be obtained compared to previous online algorithms that do not perform subsampling, thanks to the feature redundancy that often exists in high-dimensional settings.
Keywords : Matrix factorization Dictionary learning Sparsity Randomized methods Stochastic optimization Majorization-minimization
Type de document :
Communication dans un congrès
NIPS Workshop on Optimization for Machine Learning, Dec 2016, Barcelone, Spain. 2016
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01405058
Contributeur : Arthur Mensch <>
Soumis le : mercredi 30 novembre 2016 - 19:54:22
Dernière modification le : samedi 18 février 2017 - 01:14:46
Document(s) archivé(s) le : lundi 27 mars 2017 - 09:06:02

Fichiers

modl_opt.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01405058, version 1

Collections

INSMI | TDS-MACS | INRIA | CEA | LJK | DSV | UGA | LJK-GI-THOTH | LJK_GI | UNIV-PARIS-SACLAY

Relations

  • est une version de hal-01431618 - 4-page abstract early version

Citation

Arthur Mensch, Julien Mairal, Gaël Varoquaux, Bertrand Thirion. Subsampled online matrix factorization with convergence guarantees. NIPS Workshop on Optimization for Machine Learning, Dec 2016, Barcelone, Spain. 2016. <hal-01405058>

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de
la notice

615

Téléchargements du document

165

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD