A warped kernel improving robustness in Bayesian optimization via random embeddings

Mickaël Binois (1, 2, 3, 4, 5) , David Ginsbourger (6, 7) , Olivier Roustant (6, 2, 3, 4, 5)

1 Technocentre Renault [Guyancourt]
2 DEMO-ENSMSE - Département Décision en Entreprise : Modélisation, Optimisation
3 Mines Saint-Étienne MSE - École des Mines de Saint-Étienne
4 FAYOL-ENSMSE - Institut Henri Fayol
5 LIMOS - Laboratoire d'Informatique, de Modélisation et d'Optimisation des Systèmes
6 GdR MASCOT-NUM - Méthodes d'Analyse Stochastique des Codes et Traitements Numériques
7 IMSV - Institute of Mathematical Statistics and Actuarial Science [Bern]

This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the algorithm to evaluate points giving redundant information. It also alleviates constraints on bound selection for the embedded domain, thus improving the robustness, as illustrated with a test case with 25 variables and intrinsic dimension 6.

Mots clés: Gaussian Processes; High-dimensional Bayesian optimization; Warping; Kernel methods; Expected Improvement

Domaines: Optimisation et contrôle [math.OC]; Machine Learning [stat.ML]

Mickaël Binois, David Ginsbourger, Olivier Roustant. A warped kernel improving robustness in Bayesian optimization via random embeddings. Learning and Intelligent Optimization: 9th International Conference, LION 9. Revised Selected Papers, Jan 2015, Lille, France. ⟨10.1007/978-3-319-19084-6_28⟩. ⟨hal-01078003v2⟩