Best Arm Identification in Spectral Bandits

Tomáš Kocák (1) , Aurélien Garivier (1, 2)

1 UMPA-ENSL - Unité de Mathématiques Pures et Appliquées
2 LIP - Laboratoire de l'Informatique du Parallélisme

We study best-arm identification with fixed confidence in bandit models with graph smoothness constraint. We provide and analyze an efficient gradient ascent algorithm to compute the sample complexity of this problem as a solution of a non-smooth max-min problem (providing in passing a simplified analysis for the unconstrained case). Building on this algorithm, we propose an asymptotically optimal strategy. We furthermore illustrate by numerical experiments both the strategy's efficiency and the impact of the smoothness constraint on the sample complexity. Best Arm Identification (BAI) is an important challenge in many applications ranging from parameter tuning to clinical trials. It is now very well understood in vanilla bandit models, but real-world problems typically involve some dependency between arms that requires more involved models. Assuming a graph structure on the arms is an elegant practical way to encompass this phenomenon, but this had been done so far only for regret minimization. Addressing BAI with graph constraints involves delicate optimization problems for which the present paper offers a solution.

Domaines: Apprentissage [cs.LG]; Machine Learning [stat.ML]

Tomáš Kocák, Aurélien Garivier. Best Arm Identification in Spectral Bandits. IJCAI, Jul 2020, Yokohama, Japan. ⟨hal-02552568v2⟩