Skip to Main content Skip to Navigation

Developmental Reinforcement Learning through Sensorimotor Space Enlargement

Matthieu Zimmer 1 Yann Boniface 1 Alain Dutech 1
1 BISCUIT - Bio-Inspired, Situated and Cellular Unconventional Information Technologies
LORIA - AIS - Department of Complex Systems, Artificial Intelligence & Robotics
Abstract : In the framework of model-free deep reinforcement learning with continuous sensorimotor space, we propose a new type of transfer learning, inspired by the child development, where the sensorimotor space of an agent grows while it is learning a policy. To decide how the dimensions grow in our neural network based actor-critic, we add new developmental layers to the neural networks which progressively uncover some dimensions of the sensorimotor space following an Intrinsic Motivation heuristic. To mitigate the catastrophic forgetting problem, we take inspiration from the Elastic Weight Constraint to regulate the learning of the neural controller. We validate our approach using two state-of-the-art algorithms (DDPG and NFAC) on two high-dimensional environment benchmarks (Half-Cheetah and Humanoid). We show that searching first for a suboptimal solution in a subset of the parameter space, and then in the full space, is helpful to bootstrap learning algorithms, and thus reach better performances in fewer episodes.
Document type :
Conference papers
Complete list of metadatas

Cited literature [34 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01876995
Contributor : Matthieu Zimmer <>
Submitted on : Thursday, October 4, 2018 - 2:15:30 PM
Last modification on : Thursday, December 12, 2019 - 10:07:01 AM

File

main.pdf
Files produced by the author(s)

Identifiers

Citation

Matthieu Zimmer, Yann Boniface, Alain Dutech. Developmental Reinforcement Learning through Sensorimotor Space Enlargement. ICDL-EPIROB 2018 - 8th joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, Sep 2018, Tokyo, Japan. pp.1-6, ⟨10.1109/DEVLRN.2018.8761021⟩. ⟨hal-01876995v2⟩

Share

Metrics

Record views

141

Files downloads

812