Skip to Main content Skip to Navigation
Journal articles

On Bayesian index policies for sequential resource allocation

Emilie Kaufmann 1
1 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
Abstract : This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward distributions belong to a one-dimensional exponential family, for a large class of prior distributions. We also show that the Bayesian literature gives new insight on what kind of exploration rates could be used in frequentist, UCB-type algorithms. Indeed, approximations of the Bayesian optimal solution or the Finite Horizon Gittins indices provide a justification for the kl-UCB+ and kl-UCB-H+ algorithms, whose asymptotic optimality is also established.
Document type :
Journal articles
Complete list of metadata

Cited literature [38 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01251606
Contributor : Emilie Kaufmann Connect in order to contact the contributor
Submitted on : Monday, November 6, 2017 - 9:52:16 AM
Last modification on : Tuesday, April 13, 2021 - 3:13:24 AM
Long-term archiving on: : Wednesday, February 7, 2018 - 1:19:05 PM

Files

KaufmannSupplement.pdf
Files produced by the author(s)

Identifiers

`

Citation

Emilie Kaufmann. On Bayesian index policies for sequential resource allocation. Annals of Statistics, Institute of Mathematical Statistics, 2018, 46 (2), pp.842-865. ⟨10.1214/17-AOS1569⟩. ⟨hal-01251606v3⟩

Share

Metrics

Record views

449

Files downloads

353