HAL : hal-00450235, version 2

arXiv : 1001.4475

Fiche détaillée

Récupérer au format BibTeX - EndNote - TEI - RefWorks -

Journal of Machine Learning Research 12 (2011) 1655-1695

Versions disponibles :

v1 (25-01-2010)

v2 (13-04-2011)

X-Armed Bandits

Sébastien Bubeck (  Auteur correspondant ) 1, Rémi Munos 1, Gilles Stoltz 2, 3, 4, Csaba Szepesvari 5

(19/04/2011)

We consider a generalization of stochastic bandits where the set of arms, $\cX$, is allowed to be a generic measurable space and the mean-payoff function is ''locally Lipschitz'' with respect to a dissimilarity function that is known to the decision maker. Under this condition we construct an arm selection policy, called HOO (hierarchical optimistic optimization), with improved regret bounds compared to previous results for a large class of problems. In particular, our results imply that if $\cX$ is the unit hypercube in a Euclidean space and the mean-payoff function has a finite number of global maxima around which the behavior of the function is locally continuous with a known smoothness degree, then the expected regret of HOO is bounded up to a logarithmic factor by $\sqrt{n}$, i.e., the rate of growth of the regret is independent of the dimension of the space. We also prove the minimax optimality of our algorithm when the dissimilarity is a metric. Our basic strategy has quadratic computational complexity as a function of the number of time steps and does not rely on the doubling trick. We also introduce a modified strategy, which relies on the doubling trick but runs in linearithmic time. Both results are improvements with respect to previous approaches.

1 :	SEQUEL (INRIA Lille - Nord Europe)
	INRIA – CNRS : UMR8022 – CNRS : UMR8146 – Université Lille 1 - Sciences et Technologies – Université Charles de Gaulle - Lille III – Ecole Centrale de Lille
2 :	Département de Mathématiques et Applications (DMA)
	CNRS : UMR8553 – Ecole Normale Supérieure de Paris - ENS Paris
3 :	Groupement de Recherche et d'Etudes en Gestion à HEC (GREGH)
	GROUPE HEC – CNRS : UMR2959
4 :	CLASSIC (INRIA Paris - Rocquencourt)
	Ecole Normale Supérieure de Paris - ENS Paris – INRIA
5 :	Department of Computing Science
	Department of Computing Science, University of Alberta

Domaine

:

Informatique/Apprentissage Mathématiques/Statistiques Statistiques/Théorie Mathématiques/Optimisation et contrôle Sciences de l'Homme et Société/Economies et finances

Mots Clés : bandits with infinitely many arms - optimistic online optimization - regret bounds - minimax rates

hal-00450235, version 2

http://hal.archives-ouvertes.fr/hal-00450235

oai:hal.archives-ouvertes.fr:hal-00450235

Contributeur : Gilles Stoltz <>

Soumis le : Mardi 12 Avril 2011, 18:01:23

Dernière modification le : Jeudi 10 Novembre 2011, 09:01:30