Hierarchical Optimistic Region Selection driven by Curiosity

Abstract : This paper aims to take a step forwards making the term ''intrinsic motivation'' from reinforcement learning theoretically well founded, focusing on curiosity-driven learning. To that end, we consider the setting where, a fixed partition \P of a continuous space \X being given, and a process \nu defined on \X being unknown, we are asked to sequentially decide which cell of the partition to select as well as where to sample \nu in that cell, in order to minimize a loss function that is inspired from previous work on curiosity-driven learning. The loss on each cell consists of one term measuring a simple worst case quadratic sampling error, and a penalty term proportional to the range of the variance in that cell. The corresponding problem formulation extends the setting known as active learning for multi-armed bandits to the case when each arm is a continuous region, and we show how an adaptation of recent algorithms for that problem and of hierarchical optimistic sampling algorithms for optimization can be used in order to solve this problem. The resulting procedure, called Hierarchical Optimistic Region SElection driven by Curiosity (HORSE.C) is provided together with a finite-time regret analysis.
Type de document :
Pré-publication, Document de travail
2012
Liste complète des métadonnées


https://hal.archives-ouvertes.fr/hal-00740418
Contributeur : Odalric-Ambrym Maillard <>
Soumis le : jeudi 25 octobre 2012 - 07:00:08
Dernière modification le : jeudi 25 octobre 2012 - 07:00:08
Document(s) archivé(s) le : vendredi 16 décembre 2016 - 22:21:50

Fichier

horsec_nips_supplementary.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00740418, version 1

Collections

Citation

Odalric-Ambrym Maillard. Hierarchical Optimistic Region Selection driven by Curiosity. 2012. <hal-00740418>

Partager

Métriques

Consultations de
la notice

95

Téléchargements du document

91