Model-Consistent Sparse Estimation through the Bootstrap

Francis Bach 1
1 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : We consider the least-square linear regression problem with regularization by the $\ell^1$-norm, a problem usually referred to as the Lasso. In this paper, we first present a detailed asymptotic analysis of model consistency of the Lasso in low-dimensional settings. For various decays of the regularization parameter, we compute asymptotic equivalents of the probability of correct model selection. For a specific rate decay, we show that the Lasso selects all the variables that should enter the model with probability tending to one exponentially fast, while it selects all other variables with strictly positive probability. We show that this property implies that if we run the Lasso for several bootstrapped replications of a given sample, then intersecting the supports of the Lasso bootstrap estimates leads to consistent model selection. This novel variable selection procedure, referred to as the Bolasso, is extended to high-dimensional settings by a provably consistent two-step procedure.
Type de document :
Pré-publication, Document de travail
2009
Liste complète des métadonnées


https://hal.archives-ouvertes.fr/hal-00354771
Contributeur : Francis Bach <>
Soumis le : mardi 20 janvier 2009 - 20:56:36
Dernière modification le : mercredi 28 septembre 2016 - 15:33:26
Document(s) archivé(s) le : mardi 8 juin 2010 - 17:26:25

Fichiers

bolasso_hal_aos.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00354771, version 1
  • ARXIV : 0901.3202

Collections

Citation

Francis Bach. Model-Consistent Sparse Estimation through the Bootstrap. 2009. <hal-00354771>

Partager

Métriques

Consultations de
la notice

1675

Téléchargements du document

758