Model selection by resampling penalization

Sylvain Arlot 1, 2
2 WILLOW - Models of visual object recognition and scene understanding
CNRS - Centre National de la Recherche Scientifique : UMR8548, Inria Paris-Rocquencourt, DI-ENS - Département d'informatique de l'École normale supérieure
Abstract : In this paper, a new family of resampling-based penalization procedures for model selection is defined in a general framework. It generalizes several methods, including Efron's bootstrap penalization and the leave-one-out penalization recently proposed by Arlot (2008), to any exchangeable weighted bootstrap resampling scheme. In the heteroscedastic regression framework, assuming the models to have a particular structure, these resampling penalties are proved to satisfy a non-asymptotic oracle inequality with leading constant close to 1. In particular, they are asympotically optimal. Resampling penalties are used for defining an estimator adapting simultaneously to the smoothness of the regression function and to the heteroscedasticity of the noise. This is remarkable because resampling penalties are general-purpose devices, which have not been built specifically to handle heteroscedastic data. Hence, resampling penalties naturally adapt to heteroscedasticity. A simulation study shows that resampling penalties improve on V-fold cross-validation in terms of final prediction error, in particular when the signal-to-noise ratio is not large.
Document type :
Journal articles
Electronic journal of statistics , Shaker Heights, OH : Institute of Mathematical Statistics, 2009, 3, pp.557--624. 〈10.1214/08-EJS196〉
Liste complète des métadonnées

Cited literature [7 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-00262478
Contributor : Sylvain Arlot <>
Submitted on : Wednesday, June 17, 2009 - 11:28:29 AM
Last modification on : Friday, May 25, 2018 - 12:02:06 PM
Document(s) archivé(s) le : Wednesday, September 22, 2010 - 12:36:54 PM

Files

RP.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Sylvain Arlot. Model selection by resampling penalization. Electronic journal of statistics , Shaker Heights, OH : Institute of Mathematical Statistics, 2009, 3, pp.557--624. 〈10.1214/08-EJS196〉. 〈hal-00262478v2〉

Share

Metrics

Record views

912

Files downloads

331