S. H. Ahmad, M. Liu, T. Javidi, Q. Zhao, and B. Krishnamachari, Optimality of Myopic Sensing in Multichannel Opportunistic Access, IEEE Transactions on Information Theory, vol.55, issue.9, pp.4040-4050, 2009.
DOI : 10.1109/TIT.2009.2025561

P. S. Ansell, K. D. Glazebrook, J. Niño-mora, and M. O. Keeffe, Whittle's index policy for a multiclass queueing system with convex holding costs, Mathematical Methods of Operations Research, vol.57, pp.21-39, 2003.

R. Atar, C. Giat, and N. Shimkin, Rule for Many-Server Queues with Abandonment, Operations Research, vol.58, issue.5, pp.1427-1439, 2010.
DOI : 10.1287/opre.1100.0826

R. Atar, C. Giat, and N. Shimkin, On the asymptotic optimality of the c??/?? rule under??ergodic cost, Queueing Systems, vol.67, issue.2, pp.127-144, 2011.
DOI : 10.1007/s11134-010-9206-x

U. Ayesta, M. Erausquin, and P. Jacko, A modeling framework for optimizing the flow-level scheduling with time-varying channels, Performance Evaluation, vol.67, issue.11, pp.1014-1029, 2010.
DOI : 10.1016/j.peva.2010.08.015

U. Ayesta, M. Erausquin, M. Jonckheere, and I. M. Verloop, Scheduling in a Random Environment: Stability and Asymptotic Optimality, IEEE/ACM Transactions on Networking, vol.21, issue.1, pp.258-271, 2013.
DOI : 10.1109/TNET.2012.2199764

URL : https://hal.archives-ouvertes.fr/hal-01121985

U. Ayesta, P. Jacko, and V. Novak, A nearly-optimal index rule for scheduling of users with abandonment, 2011 Proceedings IEEE INFOCOM, 2011.
DOI : 10.1109/INFCOM.2011.5935122

M. Bena¨?mbena¨?m and J. Boudec, A class of mean field interaction models for computer and communication systems, Performance Evaluation, vol.65, issue.11-12, pp.823-838, 2008.
DOI : 10.1016/j.peva.2008.03.005

D. Bertsimas and J. Niño-mora, Restless Bandits, Linear Programming Relaxations, and a Primal-Dual Index Heuristic, Operations Research, vol.48, issue.1, pp.80-90, 2000.
DOI : 10.1287/opre.48.1.80.12444

P. Billingsley, Convergence of Probability Measures, 1999.
DOI : 10.1002/9780470316962

M. J. Cánovas, M. A. López, and J. Parra, On the Continuity of the Optimal Value in Parametric Linear Optimization: Stable Discretization of the Lagrangian Dual of Nonlinear Problems, Set-Valued Analysis, vol.3, issue.1, pp.69-84, 2005.
DOI : 10.1007/s11228-004-5029-4

J. G. Dai and S. He, Many-server queues with customer abandonment: A survey of diffusion and fluid approximations, Journal of Systems Science and Systems Engineering, vol.51, issue.1, pp.1-36, 2012.
DOI : 10.1007/s11518-012-5189-y

N. Ehsan and M. Liu, On the optimality of an index policy for bandwidth allocation with delayed state observation and differentiated services, IEEE INFOCOM 2004, 2004.
DOI : 10.1109/INFCOM.2004.1354606

S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence, 1986.
DOI : 10.1002/9780470316658

N. Gast and B. Gaujal, A mean field model of work stealing in large-scale systems, Proceedings of ACM SIGMETRICS, pp.13-24, 2010.
URL : https://hal.archives-ouvertes.fr/hal-00788862

J. C. Gittins, Bandit processes and dynamic allocation indices, Journal of the Royal Statistical Society. Series B, vol.41, issue.2, pp.148-177, 1979.

J. C. Gittins, Multi-Armed Bandit Allocation Indices, 1989.
DOI : 10.1002/9780470980033

J. C. Gittins, K. D. Glazebrook, and R. R. Weber, Multi-Armed Bandit Allocation Indices, 2011.
DOI : 10.1002/9780470980033

K. D. Glazebrook, D. J. Hodge, and C. Kirkbride, General notions of indexability for queueing control and asset management, The Annals of Applied Probability, vol.21, issue.3, pp.876-907, 2011.
DOI : 10.1214/10-AAP705

K. D. Glazebrook, C. Kirkbride, and J. Ouenniche, Index Policies for the Admission Control and Routing of Impatient Customers to Heterogeneous Service Stations, Operations Research, vol.57, issue.4, pp.975-989, 2009.
DOI : 10.1287/opre.1080.0632

K. D. Glazebrook and H. M. Mitchell, An index policy for a stochastic scheduling model with improving/deteriorating jobs, Naval Research Logistics, vol.25, issue.7, pp.706-721, 2002.
DOI : 10.1002/nav.10036

X. Guo, O. Hernández-lerma, and T. Prieto-rumeau, A survey of recent results on continuous-time Markov decision processes, TOP, vol.24, issue.2, pp.177-261, 2006.
DOI : 10.1007/BF02837562

D. J. Hodge and K. D. Glazebrook, Dynamic resource allocation in a multi-product make-to-stock production system, Queueing Systems, vol.41, issue.4, pp.333-364, 2011.
DOI : 10.1007/s11134-011-9217-2

URL : https://hal.archives-ouvertes.fr/hal-00803937

D. J. Hodge and K. D. Glazebrook, On the asymptotic optimality of greedy index heuristics for multi-action restless bandits, Advances in Applied Probability, 2015.

P. Jacko, Optimal Index Rules for Single Resource Allocation to Stochastic Dynamic Competitors, Proceedings of the 5th International ICST Conference on Performance Evaluation Methodologies and Tools, 2011.
DOI : 10.4108/icst.valuetools.2011.245797

M. Larrañaga, U. Ayesta, and I. M. Verloop, Dynamic fluid-based scheduling in a multi-class abandonment queue, Performance Evaluation, vol.70, issue.10, pp.841-858, 2013.
DOI : 10.1016/j.peva.2013.08.009

M. Larrañaga, U. Ayesta, and I. M. Verloop, Index policies for multi-class queues with convex holding cost and abandonments, Proceedings of ACM SIGMETRICS, 2014.

K. Liu and Q. Zhao, Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access, IEEE Transactions on Information Theory, vol.56, issue.11, pp.5547-5567, 2010.
DOI : 10.1109/TIT.2010.2068950

A. Mahajan and D. Teneketzis, Multi-Armed Bandit Problems, Foundations and Application of Sensor Management, pp.121-308, 2007.
DOI : 10.1007/978-0-387-49819-5_6

S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, 2009.

J. Niño-mora, Restless bandits, partial conservation laws and indexability, Advances in Applied Probability, vol.19, issue.01, pp.76-98, 2001.
DOI : 10.1287/moor.21.2.257

J. Niño-mora, Characterization and Computation of Restless Bandit Marginal Productivity Indices, Proceedings of the 2nd International ICST Conference on Performance Evaluation Methodologies and Tools, 2007.
DOI : 10.4108/smctools.2007.1993

J. Niño-mora, Dynamic priority allocation via restless bandit marginal productivity indices, TOP, vol.14, issue.2, pp.161-198, 2007.
DOI : 10.1007/s11750-007-0025-0

J. Niño-mora, Marginal Productivity Index Policies for Admission Control and Routing to Parallel Multi-server Loss Queues with Reneging, Lecture Notes in Computer Science, vol.4465, pp.138-149, 2007.
DOI : 10.1007/978-3-540-72709-5_15

W. Ouyang, A. Eryilmaz, and N. B. Shroff, Asymptotically optimal downlink scheduling over Markovian fading channels, 2012 Proceedings IEEE INFOCOM, 2012.
DOI : 10.1109/INFCOM.2012.6195483

D. G. Pandelis and D. Teneketzis, On the optimality of the Gittins index rule for multi-armed bandits with multiple plays, Mathematical Methods of Operations Research (ZOR), vol.50, issue.3, pp.449-461, 1999.
DOI : 10.1007/s001860050080

M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1994.
DOI : 10.1002/9780470316887

V. Raghunathan, V. Borkar, M. Cao, and P. R. Kumar, Index Policies for Real-Time Multicast Scheduling for Wireless Broadcast Systems, IEEE INFOCOM 2008, The 27th Conference on Computer Communications, 2008.
DOI : 10.1109/INFOCOM.2008.217

P. Robert, Stochastic Networks and Queues, 2003.
DOI : 10.1007/978-3-662-13052-0

A. N. Rybko and A. L. Stolyar, Ergodicity of stochastic processes describing the operation of open queuing networks. Problems of Information Transmission, pp.199-220, 1992.

H. C. Tijms, A First Course in Stochastic Models, 2003.
DOI : 10.1002/047001363X

I. M. Verloop and R. Núñez-queija, Assessing the efficiency of resource allocations in bandwidth-sharing networks, Performance Evaluation, vol.66, issue.1, pp.59-77, 2009.
DOI : 10.1016/j.peva.2008.09.002

R. R. Weber, Comments on: Dynamic priority allocation via restless bandit marginal productivity indices, TOP, vol.15, issue.2, pp.211-216, 2007.
DOI : 10.1007/s11750-007-0029-9

R. R. Weber and G. Weiss, On an index policy for restless bandits, Journal of Applied Probability, vol.25, issue.03, pp.637-648, 1990.
DOI : 10.1017/S0021900200039176

R. R. Weber and G. Weiss, Addendum to " On an index policy for restless bandits, Journal of Applied Probability, vol.23, pp.429-430, 1991.

G. Weiss, Branching Bandit Processes, Probability in the Engineering and Informational Sciences, pp.269-278, 1988.
DOI : 10.1214/aos/1176345578

P. Whittle, Arm-acquiring bandits. The Annals of Probability, pp.284-292, 1981.

P. Whittle, Restless bandits: activity allocation in a changing world, Journal of Applied Probability, vol.1, issue.A, pp.287-298, 1988.
DOI : 10.1214/aop/1176994469

P. Whittle, Optimal Control, Basics and Beyond, 1996.

X. Set, Let ? be a policy for which a unique invariant distribution exists having finite first moment. Stability of policy ? implies rate-stability