Bayesian Optimization using Deep Gaussian Processes

Ali Hebbal (1, 2) , Loic Brevault (1) , Mathieu Balesdent (1) , El-Ghazali Talbi (3, 2) , Nouredine Melab (2)

1 DTIS, ONERA, Université Paris Saclay [Palaiseau]
2 BONUS - Optimisation de grande taille et calcul large échelle
3 LIFL - Laboratoire d'Informatique Fondamentale de Lille

Bayesian Optimization using Gaussian Processes is a popular approach to deal with the optimization of expensive black-box functions. However, because of the a priori on the stationarity of the covariance matrix of classic Gaussian Processes, this method may not be adapted for non-stationary functions involved in the optimization problem. To overcome this issue, a new Bayesian Optimization approach is proposed. It is based on Deep Gaussian Processes as surrogate models instead of classic Gaussian Processes. This modeling technique increases the power of representation to capture the non-stationarity by simply considering a functional composition of stationary Gaussian Processes, providing a multiple layer structure. This paper proposes a new algorithm for Global Optimization by coupling Deep Gaussian Processes and Bayesian Optimization. The specificities of this optimization method are discussed and highlighted with academic test cases. The performance of the proposed algorithm is assessed on analytical test cases and an aerospace design optimization problem and compared to the state-of-the-art stationary and non-stationary Bayesian Optimization approaches.

Mots clés: Bayesian Optimization · Gaussian Process · Deep Gaussian Process · Non-stationary function · Global Optimization * PhD student; ONERA; DTIS

Domaines: Statistiques [stat]; Machine Learning [stat.ML]; Sciences de l'ingénieur [physics]

Ali Hebbal, Loic Brevault, Mathieu Balesdent, El-Ghazali Talbi, Nouredine Melab. Bayesian Optimization using Deep Gaussian Processes. 2020. ⟨hal-02924230⟩