Explore First, Exploit Next: The True Shape of Regret in Bandit Problems

Aurélien Garivier (1) , Pierre Ménard (1) , Gilles Stoltz (2)

1 IMT - Institut de Mathématiques de Toulouse UMR5219
2 GREGH - Groupement de Recherche et d'Etudes en Gestion à HEC

We revisit lower bounds on the regret in the case of multi-armed bandit problems. We obtain non-asymptotic bounds and provide straightforward proofs based only on well-known properties of Kullback-Leibler divergences. These bounds show that in an initial phase the regret grows almost linearly, and that the well-known logarithmic growth of the regret only holds in a final phase. The proof techniques come to the essence of the arguments used and they are deprived of all unnecessary complications.

Mots clés: information-theoretic proof techniques; multi-armed bandits; cumulative regret; non-asymptotic lower bounds

Domaines: Statistiques [math.ST]; Apprentissage [cs.LG]

Aurélien Garivier, Pierre Ménard, Gilles Stoltz. Explore First, Exploit Next: The True Shape of Regret in Bandit Problems. 2016. ⟨hal-01276324v1⟩