HAL : hal-00274327, version 2

arXiv : 0804.2937

DOI : 10.3150/10-BEJ288

Fiche détaillée

Récupérer au format BibTeX - EndNote - TEI - RefWorks -

Bernoulli 17, 2 (2011) 687-713

Versions disponibles :

v1 (18-04-2008)

v2 (08-02-2010)

Margin adaptive model selection in statistical learning

Sylvain Arlot 1, 2, Peter Bartlett 3, 4

(2011)

A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows us to take into account that learning within a small model can be much easier than in a large one. Requiring this ``strong margin adaptivity'' makes the model selection problem more challenging. We first prove, in a very general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.

1 :	Laboratoire d'informatique de l'école normale supérieure (LIENS)
	CNRS : UMR8548 – Ecole Normale Supérieure de Paris - ENS Paris
2 :	WILLOW (INRIA Rocquencourt)
	INRIA – Ecole Normale Supérieure de Paris - ENS Paris – Ecole des Ponts ParisTech – CNRS : UMR8548
3 :	Computer Science Division [Berkeley]
	University of California, Berkeley
4 :	Department of Statistics
	University of California, Berkeley

Domaine

:

Mathématiques/Statistiques Statistiques/Théorie Statistiques/Machine Learning Statistiques/Autres

Mots Clés : statistical learning – classification – empirical minimization – empirical risk minimization – margin condition – model selection – oracle inequalities – adaptivity – local Rademacher complexity

hal-00274327, version 2

http://hal.archives-ouvertes.fr/hal-00274327

oai:hal.archives-ouvertes.fr:hal-00274327

Contributeur : Sylvain Arlot <>

Soumis le : Dimanche 7 Février 2010, 19:22:44

Dernière modification le : Vendredi 29 Avril 2011, 11:50:49