Margin adaptive model selection in statistical learning

Sylvain Arlot 1, 2 Peter Bartlett 3, 4
2 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows us to take into account that learning within a small model can be much easier than in a large one. Requiring this ``strong margin adaptivity'' makes the model selection problem more challenging. We first prove, in a very general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.
Document type :
Journal articles
Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, 2011, 17 (2), pp.687-713. <10.3150/10-BEJ288>
Liste complète des métadonnées


https://hal.archives-ouvertes.fr/hal-00274327
Contributor : Sylvain Arlot <>
Submitted on : Sunday, February 7, 2010 - 7:22:44 PM
Last modification on : Wednesday, September 28, 2016 - 1:59:55 PM
Document(s) archivé(s) le : Thursday, September 23, 2010 - 11:31:52 AM

Files

margin.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Sylvain Arlot, Peter Bartlett. Margin adaptive model selection in statistical learning. Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, 2011, 17 (2), pp.687-713. <10.3150/10-BEJ288>. <hal-00274327v2>

Share

Metrics

Record views

486

Document downloads

175