Model selection in density estimation via cross-validation

Alain Celisse (1)

1 MIA-Paris - Mathématiques et Informatique Appliquées

The problem of model selection by cross-validation is addressed in the density estimation framework. Extensively used in practice, cross-validation (CV) remains poorly understood, especially in the non-asymptotic setting which is the main concern of this work. A recurrent problem with CV is the computation time it involves. This drawback is overcome here thanks to closed-form expressions for the CV estimator of the risk for a broad class of widespread estimators: projection estimators. In order to shed new lights on CV procedures with respect to the cardinality $p$ of the test set, the CV estimator is interpreted as a penalized criterion with a random penalty. For instance, the amount of penalization is shown to increase with $p$. A theoretical assessment of the CV performance is carried out thanks to two oracle inequalities applying to respectively bounded or square-integrable densities. For several collections of models, adaptivity results with respect to Hölder and Besov spaces are derived as well.

Mots clés: Density estimation; cross-validation; model selection; leave-p-out; random penalty; oracle inequality; projection estimators; adaptivity in the minimax sense; Hölder space; Besov space

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Alain Celisse. Model selection in density estimation via cross-validation. 2008. ⟨hal-00337058v2⟩