Adaptive Exploration for Continual Reinforcement Learning

Freek Stulp 1, 2
2 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : Most experiments on policy search for robotics focus on isolated tasks, where the experiment is split into two distinct phases: 1)~the learning phase, where the robot learns the task through exploration; 2)~the exploitation phase, where exploration is turned off, and the robot demonstrates its performance on the task it has learned. In this paper, we present an algorithm that enables robots to continually and autonomously alternate between these phases. We do so by combining the 'Policy Improvement with Path Integrals' direct reinforcement learning algorithm with the covariance matrix adaptation rule from the 'Cross-Entropy Method' optimization algorithm. This integration is possible because both algorithms iteratively update parameters with probability-weighted averaging. A practical advantage of the novel algorithm, called PI2-CMA, is that it alleviates the user from having to manually tune the degree of exploration. We evaluate PI2-CMA's ability to continually and autonomously tune exploration on two tasks.
Document type :
Conference papers
Liste complète des métadonnées
Contributor : Freek Stulp <>
Submitted on : Monday, February 18, 2013 - 10:54:41 AM
Last modification on : Friday, December 8, 2017 - 2:42:15 PM


  • HAL Id : hal-00789389, version 1



Freek Stulp. Adaptive Exploration for Continual Reinforcement Learning. International Conference on Intelligent Robots and Systems (IROS), 2012, Portugal. pp.0-0. ⟨hal-00789389⟩



Record views