Sparse and spurious: dictionary learning with noise and outliers

Rémi Gribonval 1 Rodolphe Jenatton 2 Francis Bach 3, 4
1 PANAMA - Parcimonie et Nouveaux Algorithmes pour le Signal et la Modélisation Audio
Inria Rennes – Bretagne Atlantique , IRISA-D5 - SIGNAUX ET IMAGES NUMÉRIQUES, ROBOTIQUE
3 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : A popular approach within the signal processing and machine learning communities consists in modelling signals as sparse linear combinations of atoms selected from a learned dictionary. While this paradigm has led to numerous empirical successes in various fields ranging from image to audio processing, there have only been a few theoretical arguments supporting these evidences. In particular, sparse coding, or sparse dictionary learning, relies on a non-convex procedure whose local minima have not been fully analyzed yet. In this paper, we consider a probabilistic model of sparse signals, and show that, with high probability, sparse coding admits a local minimum around the reference dictionary generating the signals. Our study takes into account the case of over-complete dictionaries, noisy signals, and possible outliers, thus extending previous work limited to noiseless settings and/or under-complete dictionaries. The analysis we conduct is non-asymptotic and makes it possible to understand how the key quantities of the problem, such as the coherence or the level of noise, can scale with respect to the dimension of the signals, the number of atoms, the sparsity and the number of observations.
Type de document :
Article dans une revue
IEEE Transactions on Information Theory, Institute of Electrical and Electronics Engineers, 2015, pp.22. 〈10.1109/TIT.2015.2472522〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01025503
Contributeur : Rémi Gribonval <>
Soumis le : vendredi 21 août 2015 - 15:07:28
Dernière modification le : vendredi 25 mai 2018 - 12:02:06
Document(s) archivé(s) le : mercredi 26 avril 2017 - 10:06:33

Fichiers

sparse_and_spurious.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Rémi Gribonval, Rodolphe Jenatton, Francis Bach. Sparse and spurious: dictionary learning with noise and outliers. IEEE Transactions on Information Theory, Institute of Electrical and Electronics Engineers, 2015, pp.22. 〈10.1109/TIT.2015.2472522〉. 〈hal-01025503v4〉

Partager

Métriques

Consultations de la notice

1472

Téléchargements de fichiers

317