Why V=5 is enough in V-fold cross-validation

Sylvain Arlot (1, 2) , Matthieu Lerasle (3)

1 DI-ENS - Département d'informatique - ENS Paris
2 SIERRA - Statistical Machine Learning and Parsimony
3 JAD - Laboratoire Jean Alexandre Dieudonné

This paper studies V-fold cross-validation for model selection in least-squares density estimation. The goal is to provide theoretical grounds for choosing V in order to minimize the least-squares loss of the selected estimator. We first prove a non asymptotic oracle inequality for V-fold cross-validation and its bias-corrected version (V-fold penalization). In particular, this result implies V-fold penalization is asymptotically optimal. Then, we compute the variance of V-fold cross-validation and related criteria, as well as the variance of key quantities for model selection performance. We show these variances depend on V like 1+4/(V-1) (at least in some particular cases), suggesting the performance increases much from V=2 to V=5 or 10, and then is almost constant. Overall, this explains the common advice to take V=5---at least in our setting and when the computational power is limited---, as confirmed by some simulation experiments.

Mots clés: V-fold cross-validation; leave-one-out; leave-p-out; resampling penalties; density estimation; model selection; penalization

Domaines: Statistiques [math.ST]; Théorie [stat.TH]; Apprentissage [cs.LG]; Machine Learning [stat.ML]; Méthodologie [stat.ME]

Sylvain Arlot, Matthieu Lerasle. Why V=5 is enough in V-fold cross-validation. 2014. ⟨hal-00743931v2⟩