Robust linear least squares regression

Jean-Yves Audibert 1, 2, 3 Olivier Catoni 4, 5
1 IMAGINE [Marne-la-Vallée]
LIGM - Laboratoire d'Informatique Gaspard-Monge, CSTB - Centre Scientifique et Technique du Bâtiment, ENPC - École des Ponts ParisTech
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
5 CLASSIC - Computational Learning, Aggregation, Supervised Statistical, Inference, and Classification
DMA - Département de Mathématiques et Applications - ENS Paris, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt
Abstract : We consider the problem of robustly predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. For the ridge estimator and the ordinary least squares estimator, and their variants, we provide new risk bounds of order d/n without logarithmic factor unlike some standard results, where n is the size of the training data. We also provide a new estimator with better deviations in presence of heavy-tailed noise. It is based on truncating differences of losses in a min-max framework and satisfies a d/n risk bound both in expectation and in deviations. The key common surprising factor of these results is the absence of exponential moment condition on the output distribution while achieving exponential deviations. All risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. Experimental results strongly back up our truncated min-max estimator.
Type de document :
Article dans une revue
Annals of Statistics, Institute of Mathematical Statistics, 2011, 39 (5), pp.2766-2794. 〈10.1214/11-AOS918〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-00522534
Contributeur : Jean-Yves Audibert <>
Soumis le : samedi 17 septembre 2011 - 22:11:16
Dernière modification le : jeudi 7 février 2019 - 16:03:07
Document(s) archivé(s) le : dimanche 18 décembre 2011 - 02:21:34

Fichiers

AoS_paper.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Jean-Yves Audibert, Olivier Catoni. Robust linear least squares regression. Annals of Statistics, Institute of Mathematical Statistics, 2011, 39 (5), pp.2766-2794. 〈10.1214/11-AOS918〉. 〈hal-00522534v2〉

Partager

Métriques

Consultations de la notice

1379

Téléchargements de fichiers

333