Regularity dependence of the rate of convergence of the learning curve for Gaussian process regression

Loic Le Gratiet (1, 2) , Josselin Garnier (1, 3)

1 LPMA - Laboratoire de Probabilités et Modèles Aléatoires
2 GdR MASCOT-NUM - Méthodes d'Analyse Stochastique des Codes et Traitements Numériques
3 LJLL - Laboratoire Jacques-Louis Lions

This paper deals with the speed of convergence of the learning curve in a Gaussian process regression framework. The learning curve describes the average generalization error of the Gaussian process used for the regression. More specifically, it is defined in this paper as the integral of the mean squared error over the input parameter space with respect to the probability measure of the input parameters. The main result is the proof of a theorem giving the mean squared error in function of the number of observations for a large class of kernels and for any dimension when the number of observations is large. From this result, we can deduce the asymptotic behavior of the generalization error. The presented proof generalizes previous ones that were limited to more specific kernels or to small dimensions (one or two). The result can be used to build an optimal strategy for resources allocation. This strategy is applied successfully to a nuclear safety problem.

Mots clés: Gaussian process regression; asymptotic mean squared error; learning curves; generalization erro; convergence rate

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Loic Le Gratiet, Josselin Garnier. Regularity dependence of the rate of convergence of the learning curve for Gaussian process regression. 2012. ⟨hal-00737342v3⟩