Skip to Main content Skip to Navigation
Conference papers

ELSIM: End-to-end learning of reusable skills through intrinsic motivation

Arthur Aubret 1 Laëtitia Matignon 1 Salima Hassas 1
1 SyCoSMA - Systèmes Cognitifs et Systèmes Multi-Agents
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : Taking inspiration from developmental learning, we present a novel reinforcement learning architecture which hierarchically learns and represents self-generated skills in an end-to-end way. With this architecture, an agent focuses only on task-rewarded skills while keeping the learning process of skills bottom-up. This bottom-up approach allows to learn skills that 1- are transferable across tasks, 2- improves exploration when rewards are sparse. To do so, we combine a previously defined mutual information objective with a novel curriculum learning algorithm, creating an unlimited and explorable tree of skills. We test our agent on simple gridworld environments to understand and visualize how the agent distinguishes between its skills. Then we show that our approach can scale on more difficult MuJoCo environments in which our agent is able to build a representation of skills which improve over a baseline both transfer learning and exploration when rewards are sparse.
Document type :
Conference papers
Complete list of metadata
Contributor : Laetitia Matignon <>
Submitted on : Monday, July 20, 2020 - 9:45:24 AM
Last modification on : Tuesday, March 2, 2021 - 11:50:02 AM

Links full text


  • HAL Id : hal-02902573, version 1
  • ARXIV : 2006.12903


Arthur Aubret, Laëtitia Matignon, Salima Hassas. ELSIM: End-to-end learning of reusable skills through intrinsic motivation. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), Sep 2020, Ghent, Belgium. ⟨hal-02902573⟩



Record views