Skip to Main content Skip to Navigation
Conference papers

Unsupervised Learning and Exploration of Reachable Outcome Space

Abstract : Performing Reinforcement Learning in sparse rewards settings, with very little prior knowledge, is a challenging problem since there is no signal to properly guide the learning process. In such situations, a good search strategy is fundamental. At the same time, not having to adapt the algorithm to every single problem is very desirable. Here we introduce TAXONS, a Task Agnostic eXploration of Outcome spaces through Novelty and Surprise algorithm. Based on a population-based divergent-search approach, it learns a set of diverse policies directly from high-dimensional observations, without any task-specific information. TAXONS builds a repertoire of policies while training an autoencoder on the high-dimensional observation of the final state of the system to build a low-dimensional outcome space. The learned outcome space, combined with the reconstruction error, is used to drive the search for new policies. Results show that TAXONS can find a diverse set of controllers, covering a good part of the ground-truth outcome space, while having no information about such space.
Complete list of metadata
Contributor : Giuseppe Paolo <>
Submitted on : Monday, September 28, 2020 - 3:52:10 PM
Last modification on : Tuesday, March 30, 2021 - 5:47:56 PM

Links full text



Giuseppe Paolo, Alban Laflaquière, Alexandre Coninx, Stephane Doncieux. Unsupervised Learning and Exploration of Reachable Outcome Space. IEEE International Conference on Robotics and Automation (ICRA), 2020, Paris, France. ⟨10.1109/icra40945.2020.9196819⟩. ⟨hal-02951255⟩



Record views