Kernels over Sets of Finite Sets using RKHS Embeddings, with Application to Bayesian (Combinatorial) Optimization

Poompol Buathong (1) , David Ginsbourger (2, 3, 4) , Tipaluck Krityakierne (1, 5)

1 Mahidol University [Bangkok]
2 Idiap - Idiap Research Institute
3 IMSV - Institute of Mathematical Statistics and Actuarial Science [Bern]
4 GdR MASCOT-NUM - Méthodes d'Analyse Stochastique des Codes et Traitements Numériques
5 Centre of Excellence in Mathematics

We focus on kernel methods for set-valued inputs and their application to Bayesian set optimization , notably combinatorial optimization. We introduce a class of (strictly) positive definite kernels that relies on Reproducing Kernel Hilbert Space embeddings, and successfully generalizes "double sum" set kernels recently considered in Bayesian set optimization , which turn out to be unsuitable for combinatorial optimization. The proposed class of kernels, for which we provide theoretical guarantees, essentially consists in applying an outer kernel on top of the canoni-cal distance induced by a double sum kernel. Proofs of theoretical results about considered kernels are complemented by a few practical-ities regarding hyperparameter fitting. We furthermore demonstrate the applicability of our approach in prediction and optimization tasks, relying both on toy examples and on two test cases from mechanical engineering and hydrogeology, respectively. Experimental results illustrate the added value of the approach and open new perspectives in prediction and sequential design with set inputs.

Domaines: Modélisation et simulation; Autres [stat.ML]; Statistiques [stat]; Intelligence artificielle [cs.AI]; Mathématique discrète [cs.DM]; Apprentissage [cs.LG]; Recherche opérationnelle [math.OC]; Optimisation et contrôle [math.OC]; Statistiques [math.ST]

Poompol Buathong, David Ginsbourger, Tipaluck Krityakierne. Kernels over Sets of Finite Sets using RKHS Embeddings, with Application to Bayesian (Combinatorial) Optimization. 2019. ⟨hal-02309743⟩