Optimal upper and lower bounds for the true and empirical excess risks in heteroscedastic least-squares regression

Adrien Saumard (1, 2)

1 IRMAR - Institut de Recherche Mathématique de Rennes
2 LTCI - Laboratoire Traitement et Communication de l'Information

We consider the estimation of a bounded regression function with nonparametric heteroscedastic noise. We are interested by the true and empirical excess risks of the least-squares estimator on a …nite-dimensional vector space. For these quantities, we give upper and lower bounds in probability that are optimal at the …rst order. Moreover, these bounds show the equivalence between the true and empirical excess risks when, among other things, the least-squares estimator is consistent in sup-norm towards the projection of the regression function onto the considered model. Consistency in sup-norm is then proved for suitable histogram models and more general models of piecewise polynomials that are endowed with a localized basis structure.

Mots clés: Least-squares regression; Heteroscedasticity; Excess risk; Lower bounds; Empirical process; M-estimation

Domaines: Autres [stat.ML]

Adrien Saumard. Optimal upper and lower bounds for the true and empirical excess risks in heteroscedastic least-squares regression. 2010. ⟨hal-00512304v1⟩