HAL : hal-00408867, version 2

DOI : 10.1109/JSTSP.2010.2058091

Fiche détaillée

Récupérer au format BibTeX - EndNote - TEI - RefWorks -

Selected Topics in Signal Processing, IEEE Journal of 5, 1 (2010) 68 - 76

Versions disponibles :

v1 (03-08-2009)

v2 (12-04-2010)

Optimally Sensing a Single Channel Without Prior Information: The Tiling Algorithm and Regret Bounds

Sarah Filippi (  Auteur correspondant ) 1, Olivier Cappé 1, Aurélien Garivier 1

(01/02/2010)

We consider the task of optimally sensing a two-state Markovian channel with an observation cost and without any prior information regarding the channel's transition probabilities. This task is of interest in the field of cognitive radio as a model for opportunistic access to a communication network by a secondary user. The optimal sensing problem may be cast into the framework of model-based reinforcement learning in a specific class of Partially Observable Markov Decision Processes (POMDPs). We propose the Tiling Algorithm, an original method aimed at reaching an optimal tradeoff between the exploration (or estimation) and exploitation requirements. It is shown that this algorithm achieves finite horizon regret bounds that are as good as those recently obtained for multi-armed bandits and finite-state Markov Decision Processes (MDPs).

1 :	Laboratoire traitement et communication de l'information (LTCI)
	CNRS : UMR5141 – Institut Télécom – Télécom ParisTech

Domaine

:

Statistiques/Machine Learning Informatique/Apprentissage Informatique/Intelligence artificielle Informatique/Réseaux et télécommunications

Mots Clés : Cognitive Radio – Opportunistic Channel Access – POMDPs – Regret Bounds – Reinforcement learning – Restless Bandit.

hal-00408867, version 2

http://hal.archives-ouvertes.fr/hal-00408867

oai:hal.archives-ouvertes.fr:hal-00408867

Contributeur : Sarah Filippi <>

Soumis le : Mardi 23 Mars 2010, 10:48:59

Dernière modification le : Mercredi 21 Septembre 2011, 11:57:10