An analytic comparison of regularization methods for Gaussian Processes - Archive ouverte HAL Accéder directement au contenu
Rapport (Rapport De Recherche) Année : 2016

An analytic comparison of regularization methods for Gaussian Processes

Résumé

Gaussian Processes (GPs) are often used to predict the output of a parameterized deterministic experiment. They have many applications in the field of Computer Experiments, in particular to perform sensitivity analysis, adaptive design of experiments and global optimization. Nearly all of the applications of GPs to Computer Experiments require the inversion of a covariance matrix. Because this matrix is often ill-conditioned, regularization techniques are required. Today, there is still a need to better regularize GPs. The two most classical regularization methods to avoid degeneracy of the covariance matrix are i) pseudoinverse (PI) and ii) adding a small positive constant to the main diagonal, i.e., the case of noisy observations. Herein, we will refer to the second regularization technique with a slight abuse of language as nugget. This paper provides algebraic calculations which allow comparing PI and nugget regularizations. It is proven that pseudoinverse regularization averages the output values and makes the variance null at redundant points. On the opposite, nugget regularization lacks interpolation properties but preserves a non-zero variance at every point. However, these two regularization techniques become similar as the nugget value decreases. A distribution-wise GP is introduced which interpolates Gaussian distributions instead of data points and mitigates the drawbacks of pseudoinverse and nugget regularized GPs. Finally, data-model discrepancy is discussed and serves as a guide for choosing a regularization technique.
Fichier principal
Vignette du fichier
regularization_kriging_2.pdf (639.93 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01264192 , version 1 (02-02-2016)
hal-01264192 , version 2 (21-03-2016)
hal-01264192 , version 3 (06-06-2016)
hal-01264192 , version 4 (05-05-2017)

Identifiants

Citer

Hossein Mohammadi, Rodolphe Le Riche, Nicolas Durrande, Eric Touboul, Xavier Bay. An analytic comparison of regularization methods for Gaussian Processes. [Research Report] Ecole Nationale Supérieure des Mines de Saint-Etienne; LIMOS. 2016. ⟨hal-01264192v3⟩
854 Consultations
1049 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More