Regret Bounds for Opportunistic Channel Access

Sarah Filippi (1) , Olivier Cappé (1) , Aurélien Garivier (1)

1 LTCI - Laboratoire Traitement et Communication de l'Information

We consider the task of opportunistic channel access in a primary system composed of independent Gilbert-Elliot channels where the secondary (or opportunistic) user does not dispose of a priori information regarding the statistical characteristics of the system. It is shown that this problem may be cast into the framework of model-based learning in a specific class of Partially Observed Markov Decision Processes (POMDPs) for which we introduce an algorithm aimed at striking an optimal tradeoff between the exploration (or estimation) and exploitation requirements. We provide finite horizon regret bounds for this algorithm as well as a numerical evaluation of its performance in the single channel model as well as in the case of stochastically identical channels.

Mots clés: opportunistic channel access; regret bounds; model-based reinforcement learning; POMDP; cognitive radio

Domaines: Machine Learning [stat.ML]; Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]; Réseaux et télécommunications [cs.NI]

Sarah Filippi, Olivier Cappé, Aurélien Garivier. Regret Bounds for Opportunistic Channel Access. 2009. ⟨hal-00408867v1⟩