Emergent Proximo-Distal Maturation through Adaptive Exploration

Abstract : Life-long robot learning in the high-dimensional real world requires guided and structured exploration mechanisms. In this developmental context, we investigate here the use of the recently proposed PI2-CMAES episodic reinforcement learning algorithm, which is able to learn high-dimensional motor tasks through adaptive control of exploration. By studying PI2-CMAES in a reaching task on a simulated arm, we observe two developmental properties. First, we show how PI2-CMAES autonomously and continuously tunes the global exploration/exploitation trade-off, allowing it to re-adapt to changing tasks. Second, we show how PI2-CMAES spontaneously self-organizes a maturational structure whilst exploring the degrees-of-freedom (DOFs) of the motor space. In particular, it automatically demonstrates the so-called proximo-distal maturation observed in humans: after first freezing distal DOFs while exploring predominantly the most proximal DOF, it progressively frees exploration in DOFs along the proximo-distal body axis. These emergent properties suggest the use of PI2-CMAES as a general tool for studying reinforcement learning of skills in life-long developmental learning contexts.
Document type :
Conference papers
Liste complète des métadonnées

Contributor : Freek Stulp <>
Submitted on : Monday, February 18, 2013 - 10:59:36 AM
Last modification on : Wednesday, November 29, 2017 - 3:53:16 PM


  • HAL Id : hal-00789393, version 1



Freek Stulp, Pierre-Yves Oudeyer. Emergent Proximo-Distal Maturation through Adaptive Exploration. International Conference on Development and Learning (ICDL), 2012, United States. pp.0-0. ⟨hal-00789393⟩



Record views