Skip to Main content Skip to Navigation
Conference papers

Developmental Reinforcement Learning through Sensorimotor Space Enlargement

Matthieu Zimmer 1 Yann Boniface 1 Alain Dutech 1 
1 BISCUIT - Bio-Inspired, Situated and Cellular Unconventional Information Technologies
LORIA - AIS - Department of Complex Systems, Artificial Intelligence & Robotics
Abstract : In the framework of model-free deep reinforcement learning with continuous sensorimotor space, we propose a new type of transfer learning, inspired by the child development, where the sensorimotor space of an agent grows while it is learning a policy. To decide how the dimensions grow in our neural network based actor-critic, we add new developmental layers to the neural networks which progressively uncover some dimensions of the sensorimotor space following an Intrinsic Motivation heuristic. To mitigate the catastrophic forgetting problem, we take inspiration from the Elastic Weight Constraint to regulate the learning of the neural controller. We validate our approach using two state-of-the-art algorithms (DDPG and NFAC) on two high-dimensional environment benchmarks (Half-Cheetah and Humanoid). We show that searching first for a suboptimal solution in a subset of the parameter space, and then in the full space, is helpful to bootstrap learning algorithms, and thus reach better performances in fewer episodes.
Document type :
Conference papers
Complete list of metadata

Cited literature [34 references]  Display  Hide  Download
Contributor : Matthieu Zimmer Connect in order to contact the contributor
Submitted on : Thursday, October 4, 2018 - 2:15:30 PM
Last modification on : Wednesday, November 3, 2021 - 7:57:00 AM


Files produced by the author(s)



Matthieu Zimmer, Yann Boniface, Alain Dutech. Developmental Reinforcement Learning through Sensorimotor Space Enlargement. ICDL-EPIROB 2018 - 8th joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, Sep 2018, Tokyo, Japan. pp.1-6, ⟨10.1109/DEVLRN.2018.8761021⟩. ⟨hal-01876995v2⟩



Record views


Files downloads