Scaling Bayesian Optimization up to Higher Dimensions: a Review and Comparison of Recent Algorithms

Benoît Choffin (1, 2) , Naonori Ueda (3, 4)

1 LRI - Laboratoire de Recherche en Informatique
2 MODHEL - LRI - Modélisation hétérogène (LRI)
3 RIKEN AIP - RIKEN Center for Advanced Intelligence Project [Tokyo]
4 NTT Communication Science Laboratories

Bayesian optimization is known to be a method of choice when it comes to solving optimization problems involving black-box, non-convex and low-dimensional functions in a few iterations. Yet, how to scale this method up to higher dimensions is a challenging and still unsolved research issue. In this paper, we first present and structure recent axes of research addressing this topic. We then experimentally compare three selected high-dimensional Bayesian optimization algorithms to random search on diverse high-dimensional functions. Our results suggest that no algorithm consistently outperforms the others across all types of difficulties encountered and that random search is in general very competitive, confirming recent research results.

Mots clés: Bayesian optimization; High dimension

Domaines: Machine Learning [stat.ML]

Benoît Choffin, Naonori Ueda. Scaling Bayesian Optimization up to Higher Dimensions: a Review and Comparison of Recent Algorithms. 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), Sep 2018, Aalborg, Denmark. ⟨10.1109/MLSP.2018.8517011⟩. ⟨hal-02265260⟩