Forced-exploration free Strategies for Unimodal Bandits

Hassan Saber (1, 2) , Pierre Ménard (1, 2) , Odalric-Ambrym Maillard (1, 2)

1 SEQUEL - Sequential Learning
2 Scool - Scool

We consider a multi-armed bandit problem specified by a set of Gaussian or Bernoulli distributions endowed with a unimodal structure. Although this problem has been addressed in the literature (Combes and Proutiere, 2014), the state-of-the-art algorithms for such structure make appear a forced-exploration mechanism. We introduce IMED-UB, the first forced-exploration free strategy that exploits the unimodal-structure, by adapting to this setting the Indexed Minimum Empirical Divergence (IMED) strategy introduced by Honda and Takemura (2015). This strategy is proven optimal. We then derive KLUCB-UB, a KLUCB version of IMED-UB, which is also proven optimal. Owing to our proof technique, we are further able to provide a concise finite-time analysis of both strategies in an unified way. Numerical experiments show that both IMED-UB and KLUCB-UB perform similarly in practice and outperform the state-of-the-art algorithms.

Mots clés: Structured Bandits; Indexed Minimum Empirical Divergence; Optimal Strategy

Domaines: Théorie de l'information [cs.IT]; Machine Learning [stat.ML]

Hassan Saber, Pierre Ménard, Odalric-Ambrym Maillard. Forced-exploration free Strategies for Unimodal Bandits. 2020. ⟨hal-02883907⟩