HAL : hal-00414774, version 2

arXiv : 0909.1884

Fiche détaillée

Récupérer au format BibTeX - EndNote - TEI - RefWorks -

NIPS 2009 - Advances in Neural Information Processing Systems, Vancouver : Canada (2009)

Versions disponibles :

v1 (10-09-2009)

v2 (13-09-2011)

Data-driven calibration of linear estimators with minimal penalties

Sylvain Arlot 1, 2, Francis Bach 1, 2

(12/2009)

This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kernel learning. We propose a new algorithm which first estimates consistently the variance of the noise, based upon the concept of minimal penalty, which was previously introduced in the context of model selection. Then, plugging our variance estimate in Mallows' $C_L$ penalty is proved to lead to an algorithm satisfying an oracle inequality. Simulation experiments with kernel ridge regression and multiple kernel learning show that the proposed algorithm often improves significantly existing calibration procedures such as generalized cross-validation.

1 :	Laboratoire d'informatique de l'école normale supérieure (LIENS)
	CNRS : UMR8548 – Ecole Normale Supérieure de Paris - ENS Paris
2 :	SIERRA (INRIA Paris - Rocquencourt)
	INRIA : PARIS - ROCQUENCOURT – Ecole Normale Supérieure de Paris - ENS Paris – CNRS : UMR8548

Domaine

:

Mathématiques/Statistiques Statistiques/Théorie Statistiques/Autres Statistiques/Méthodologie

Mots Clés : Data-driven calibration – Non-parametric regression – Model selection by penalization – Minimal penalty – Kernel ridge regression – Multiple kernel learning

hal-00414774, version 2

http://hal.archives-ouvertes.fr/hal-00414774

oai:hal.archives-ouvertes.fr:hal-00414774

Contributeur : Sylvain Arlot <>

Soumis le : Lundi 12 Septembre 2011, 16:57:09

Dernière modification le : Mercredi 14 Septembre 2011, 11:14:34