The degrees of freedom of penalized l1 minimization

Charles H Dossal (1) , Maher Kachour (2) , Jalal M. Fadili (3) , Gabriel Peyré (4) , Christophe Chesneau (5)

1 IMB - Institut de Mathématiques de Bordeaux
2 GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen
3 Equipe Image - Laboratoire GREYC - UMR6072
4 CEREMADE - CEntre de REcherches en MAthématiques de la DEcision
5 LMNO - Laboratoire de Mathématiques Nicolas Oresme

In this paper, we investigate the degrees of freedom (df) of penalized l1 minimization (also known as the Lasso) for linear regression models. We give a closed-form expression of the degrees of freedom of the Lasso response. Namely, we show that for any given Lasso regularization parameter$ \lambda$ and any observed data y belongs to a set of full measure, the cardinal of the support of a particular solution of the Lasso problem is an unbiased estimator of the degrees of freedom of the Lasso response. This work is achieved without any assumption on the uniqueness of the Lasso solution. Thus, our result remains true for both the underdetermined and the overdetermined case studied originally in Zou et al.. We also prove that a key result in Zou et al. is not true by providing a simple counterexample. An effective estimator of the number of degrees of freedom may have several applications including an objectively guided choice of the regularization parameter in the Lasso through the SURE framework.

Mots clés: Lasso; model selection criteria; degrees of freedom; SURE

Domaines: Statistiques [math.ST]; Théorie [stat.TH]; Théorie de l'information et codage [math.IT]; Théorie de l'information [cs.IT]

Charles H Dossal, Maher Kachour, Jalal M. Fadili, Gabriel Peyré, Christophe Chesneau. The degrees of freedom of penalized l1 minimization. 2011. ⟨hal-00638417v1⟩