Estimating prediction error: Cross-validation vs accumulated prediction error

Jenny Häggström (1) , Xavier de Luna (2)

1 Dept of Statistics
2 Dept of Statistics

We study the validation of prediction rules such as regression models and classification algorithms through two out-of-sample strategies, cross-validation and accumulated prediction error. We use the framework of Efron (1983) where measures of prediction errors are defined as sample averages of expected errors and show through exact finite sample calculations that cross-validation and accumulated prediction error yield different smoothing parameter choices in non-parametric regression. The difference in choice does not vanish as sample size increases.

Mots clés: Physical Sciences

Jenny Häggström, Xavier de Luna. Estimating prediction error: Cross-validation vs accumulated prediction error. Communications in Statistics - Simulation and Computation, 2010, 39 (05), pp.880-898. ⟨10.1080/03610911003650409⟩. ⟨hal-00591699⟩