Structured sparsity through convex optimization

Francis Bach (1, 2) , Rodolphe Jenatton (1, 2) , Julien Mairal (3) , Guillaume Obozinski (1, 2)

1 LIENS - Laboratoire d'informatique de l'école normale supérieure
2 SIERRA - Statistical Machine Learning and Parsimony
3 Department of Statistics [Berkeley]

Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. While naturally cast as a combinatorial optimization problem, variable or feature selection admits a convex relaxation through the regularization by the $\ell_1$-norm. In this paper, we consider situations where we are not only interested in sparsity, but where some structural prior knowledge is available as well. We show that the $\ell_1$-norm can then be extended to structured norms built on either disjoint or overlapping groups of variables, leading to a flexible framework that can deal with various structures. We present applications to supervised learning in the context of non-linear variable selection, and to unsupervised learning, for structured sparse principal component analysis, and hierarchical dictionary learning.

Domaines: Apprentissage [cs.LG]; Machine Learning [stat.ML]; Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Francis Bach, Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski. Structured sparsity through convex optimization. 2011. ⟨hal-00621245v1⟩