The degrees of freedom of the Lasso for general design matrix
Résumé
In this paper, we investigate the degrees of freedom ($df$) of penalized $\ell_1$ minimization (also known as the Lasso) for linear regression models. We give a closed-form expression of the degrees of freedom of the Lasso response. Namely, we show that for any given Lasso regularization parameter $\lambda$ and any observed data $y$ belongs to a set of full measure, the cardinality of the support of a particular solution of the Lasso problem is an unbiased estimator of the degrees of freedom of the Lasso response. This work is achieved without any assumption on the uniqueness of the Lasso solution. Thus, our result remains true for both the underdetermined and the overdetermined case studied originally in \cite{zou}. We also show, by providing a simple counterexample, that although the $df$ theorem of \cite{zou} is correct, their proof contains a flaw since their divergence formula holds on a different set of a full measure than the one that they claim. An effective estimator of the number of degrees of freedom may have several applications including an objectively guided choice of the regularization parameter in the Lasso through the SURE framework. as we illustrate in some numerical simulations.
Origine : Fichiers produits par l'(les) auteur(s)