HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Asymptotically optimal priority policies for indexable and non-indexable restless bandits

Ina Maria Verloop 1, 2, *
* Corresponding author
1 IRIT-RMESS - Réseaux, Mobiles, Embarqués, Sans fil, Satellites
IRIT - Institut de recherche en informatique de Toulouse
Abstract : We study the asymptotic optimal control of multi-class restless bandits. A restless bandit is a controllable stochastic process whose state evolution depends on whether or not the bandit is made active. Since finding the optimal control is typically intractable, we propose a class of priority policies that are proved to be asymptotically optimal under a global attractor property and a technical condition. We consider both a fixed population of bandits as well as a dynamic population where bandits can depart and arrive. As an example of a dynamic population of bandits, we analyze a multi-class M/M/S+M queue for which we show asymptotic optimality of an index policy.We combine fluid-scaling techniques with linear programming results to prove that when bandits are indexable, Whittle's index policy is included in our class of priority policies. We thereby generalize a result of Weber and Weiss (1990) about asymptotic optimality of Whittle's index policy to settings with (i) several classes of bandits, (ii) arrivals of new bandits, and (iii) multiple actions. Indexability of the bandits is not required for our results to hold. For non-indexable bandits we describe how to select priority policies from the class of asymptotically optimal policies and present numerical evidence that, outside the asymptotic regime, the performance of our proposed priority policies is nearly optimal.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

Cited literature [50 references]  Display  Hide  Download

Contributor : Ina Maria Verloop Connect in order to contact the contributor
Submitted on : Monday, February 29, 2016 - 2:44:09 PM
Last modification on : Tuesday, October 19, 2021 - 2:24:13 PM
Long-term archiving on: : Monday, May 30, 2016 - 3:20:16 PM


HAL_RBP (1).pdf
Files produced by the author(s)


  • HAL Id : hal-00743781, version 6


Ina Maria Verloop. Asymptotically optimal priority policies for indexable and non-indexable restless bandits. 2014. ⟨hal-00743781v6⟩



Record views


Files downloads