Optimal model selection in heteroscedastic regression using piecewise polynomials

Adrien Saumard (1)

1 SELECT - Model selection in statistical learning

We consider the estimation of a regression function with random design and heteroscedastic noise in a non-parametric setting. More precisely, we address the problem of characterizing the optimal penalty when the regression function is estimated by using a penalized least-squares model selection method. In this context, we show the existence of a minimal penalty, de…ned to be the maximum level of penalization under which the model selection procedure totally misbehaves. Moreover, the optimal penalty is shown to be twice the minimal one and to satisfy a nonasymptotic pathwise oracle inequality with leading constant almost one. When the shape of the optimal penalty is known, this allows to apply the so-called slope heuristics initially proposed by Birgé and Massart (2007), which further provides with a data-driven calibration of penalty procedure. Finally, the use of the results obtained in [30], considering the least-squares estimation of a regression function on a …xed …nite-dimensional linear model, allows us to go beyond the case of histogram models, which is already treated by Arlot and Massart in [6].

Domaines: Machine Learning [stat.ML]

Adrien Saumard. Optimal model selection in heteroscedastic regression using piecewise polynomials. 2010. ⟨hal-00512306v2⟩