X-Armed Bandits

Sébastien Bubeck 1, * Rémi Munos 1 Gilles Stoltz 2, 3, 4 Csaba Szepesvari 5
* Corresponding author
1 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
4 CLASSIC - Computational Learning, Aggregation, Supervised Statistical, Inference, and Classification
DMA - Département de Mathématiques et Applications - ENS Paris, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt
Abstract : We consider a generalization of stochastic bandits where the set of arms, $\cX$, is allowed to be a generic measurable space and the mean-payoff function is ''locally Lipschitz'' with respect to a dissimilarity function that is known to the decision maker. Under this condition we construct an arm selection policy, called HOO (hierarchical optimistic optimization), with improved regret bounds compared to previous results for a large class of problems. In particular, our results imply that if $\cX$ is the unit hypercube in a Euclidean space and the mean-payoff function has a finite number of global maxima around which the behavior of the function is locally continuous with a known smoothness degree, then the expected regret of HOO is bounded up to a logarithmic factor by $\sqrt{n}$, i.e., the rate of growth of the regret is independent of the dimension of the space. We also prove the minimax optimality of our algorithm when the dissimilarity is a metric. Our basic strategy has quadratic computational complexity as a function of the number of time steps and does not rely on the doubling trick. We also introduce a modified strategy, which relies on the doubling trick but runs in linearithmic time. Both results are improvements with respect to previous approaches.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-00450235
Contributor : Gilles Stoltz <>
Submitted on : Tuesday, April 12, 2011 - 6:01:23 PM
Last modification on : Thursday, June 27, 2019 - 1:36:06 PM
Long-term archiving on : Wednesday, July 13, 2011 - 2:25:50 AM

Files

HOO-arXiv.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00450235, version 2
  • ARXIV : 1001.4475

Citation

Sébastien Bubeck, Rémi Munos, Gilles Stoltz, Csaba Szepesvari. X-Armed Bandits. Journal of Machine Learning Research, Microtome Publishing, 2011, 12, pp.1655-1695. ⟨hal-00450235v2⟩

Share

Metrics

Record views

1244

Files downloads

556