Convex Relaxation for Combinatorial Penalties

Guillaume Obozinski (1, 2) , Francis Bach (1, 2)

1 SIERRA - Statistical Machine Learning and Parsimony
2 LIENS - Laboratoire d'informatique de l'école normale supérieure

In this paper, we propose an unifying view of several recently proposed structured sparsity-inducing norms. We consider the situation of a model simultaneously (a) penalized by a set- function de ned on the support of the unknown parameter vector which represents prior knowledge on supports, and (b) regularized in Lp-norm. We show that the natural combinatorial optimization problems obtained may be relaxed into convex optimization problems and introduce a notion, the lower combinatorial envelope of a set-function, that characterizes the tightness of our relaxations. We moreover establish links with norms based on latent representations including the latent group Lasso and block-coding, and with norms obtained from submodular functions.

Mots clés: sparsity; structured sparsity; submodular function; relaxation; set cover; set function; regularization; support recovery

Domaines: Machine Learning [stat.ML]

Guillaume Obozinski, Francis Bach. Convex Relaxation for Combinatorial Penalties. [Research Report] INRIA. 2012. ⟨hal-00694765⟩