HAL : hal-00638417, version 3

Fiche détaillée

Récupérer au format BibTeX - EndNote - TEI - RefWorks -

Versions disponibles :

v1 (04-11-2011)

v2 (27-03-2012)

v3 (29-05-2012)

The degrees of freedom of the Lasso for general design matrix

Charles Dossal 1, Maher Kachour 2, Jalal Fadili 2, Gabriel Peyré 3, Christophe Chesneau 4

(31/08/2011)

In this paper, we investigate the degrees of freedom ($\dof$) of penalized $\ell_1$ minimization (also known as the Lasso) for linear regression models. We give a closed-form expression of the $\dof$ of the Lasso response. Namely, we show that for any given Lasso regularization parameter $\lambda$ and any observed data $y$ belonging to a set of full (Lebesgue) measure, the cardinality of the support of a particular solution of the Lasso problem is an unbiased estimator of the degrees of freedom. This is achieved without the need of uniqueness of the Lasso solution. Thus, our result holds true for both the underdetermined and the overdetermined case, where the latter was originally studied in \cite{zou}. We also show, by providing a simple counterexample, that although the $\dof$ theorem of \cite{zou} is correct, their proof contains a flaw since their divergence formula holds on a different set of a full measure than the one that they claim. An effective estimator of the number of degrees of freedom may have several applications including an objectively guided choice of the regularization parameter in the Lasso through the $\sure$ framework. Our theoretical findings are illustrated through several numerical simulations.

1 :	Institut de Mathématiques de Bordeaux (IMB)
	CNRS : UMR5251 – Université Sciences et Technologies - Bordeaux I – Université Victor Segalen - Bordeaux II
2 :	Groupe de Recherche en Informatique, Image, Automatique et Instrumentation de Caen (GREYC)
	CNRS : UMR6072 – Université de Caen Basse-Normandie – Ecole Nationale Supérieure d'Ingénieurs de Caen
3 :	CEntre de REcherches en MAthématiques de la DEcision (CEREMADE)
	CNRS : UMR7534 – Université Paris IX - Paris Dauphine
4 :	Laboratoire de Mathématiques Nicolas Oresme (LMNO)
	CNRS : UMR6139 – Université de Caen Basse-Normandie

Équipe de recherche : image

Domaine

:

Mathématiques/Statistiques Statistiques/Théorie Mathématiques/Théorie de l'information et codage Informatique/Théorie de l'information et codage

Mots Clés : Lasso – model selection criteria – degrees of freedom – SURE

hal-00638417, version 3

http://hal.archives-ouvertes.fr/hal-00638417

oai:hal.archives-ouvertes.fr:hal-00638417

Contributeur : Jalal Fadili <>

Soumis le : Lundi 28 Mai 2012, 22:42:36

Dernière modification le : Mardi 29 Mai 2012, 08:43:52