Skip to Main content Skip to Navigation
Conference papers

CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

Cédric Colas 1 Pierre Fournier 2 Olivier Sigaud 2 Mohamed Chetouani 2, 3 Pierre-Yves Oudeyer 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
3 PIROS - Perception, Interaction, Robotique sociales
ISIR - Institut des Systèmes Intelligents et de Robotique
Abstract : In open-ended environments, autonomous learning agents must set their own goals and build their own curriculum through an intrinsically motivatedexploration. They may consider a large diversity of goals, aiming to discover what is controllable in their environments, and what is not. Becausesome goals might prove easy and some impossible, agents must actively select which goal to practice at any moment, to maximize their overallmastery on the set of learnable goals. This paper proposes CURIOUS, an algorithm that leverages 1) a modular Universal Value Function Approxi-mator with hindsight learning to achieve a diversity of goals of different kinds within a unique policy and 2) an automated curriculum learningmechanism that biases the attention of the agent towards goals maximizing the absolute learning progress. Agents focus sequentially on goals ofincreasing complexity, and focus back on goals that are being forgotten. Experiments conducted in a new modular-goal robotic environment showthe resulting developmental self-organization of a learning curriculum, and demonstrate properties of robustness to distracting goals, forgetting andchanges in body properties.
Complete list of metadata

Cited literature [30 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01934921
Contributor : Cédric Colas Connect in order to contact the contributor
Submitted on : Tuesday, January 7, 2020 - 11:45:37 AM
Last modification on : Tuesday, November 30, 2021 - 3:46:16 AM

File

CURIOUS_ICML_2019.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01934921, version 2

Citation

Cédric Colas, Pierre Fournier, Olivier Sigaud, Mohamed Chetouani, Pierre-Yves Oudeyer. CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning. ICML 2019 - Thirty-sixth International Conference on Machine Learning, Jun 2019, Long Beach, United States. ⟨hal-01934921v2⟩

Share

Metrics

Record views

1029

Files downloads

188