Developmental Reinforcement Learning through Sensorimotor Space Enlargement - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Developmental Reinforcement Learning through Sensorimotor Space Enlargement

Résumé

In the framework of model-free deep reinforcement learning with continuous sensorimotor space, we propose a new type of transfer learning, inspired by the child development, where the sensorimotor space of an agent grows while it is learning a policy. To decide how the dimensions grow in our neural network based actor-critic, we add new developmental layers to the neural networks which progressively uncover some dimensions of the sensorimotor space following an Intrinsic Motivation heuristic. To mitigate the catastrophic forgetting problem, we take inspiration from the Elastic Weight Constraint to regulate the learning of the neural controller. We validate our approach using two state-of-the-art algorithms (DDPG and NFAC) on two high-dimensional environment benchmarks (Half-Cheetah and Humanoid). We show that searching first for a suboptimal solution in a subset of the parameter space, and then in the full space, is helpful to bootstrap learning algorithms, and thus reach better performances in fewer episodes.
Fichier principal
Vignette du fichier
main.pdf (1 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01876995 , version 1 (19-09-2018)
hal-01876995 , version 2 (04-10-2018)

Identifiants

Citer

Matthieu Zimmer, Yann Boniface, Alain Dutech. Developmental Reinforcement Learning through Sensorimotor Space Enlargement. ICDL-EPIROB 2018 - 8th joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, Sep 2018, Tokyo, Japan. pp.1-6, ⟨10.1109/DEVLRN.2018.8761021⟩. ⟨hal-01876995v2⟩
203 Consultations
861 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More