Imitation Learning with Non-Parametric Regression

Maarten Vaandrager (1) , Robert Babuska (1) , Lucian Busoniu (2) , Gabriel Lopes (1)

1 DCSC - Delft Center for Systems and Control [Delft]
2 CRAN - Centre de Recherche en Automatique de Nancy

Humans are very fast learners. Yet, we rarely learn a task completely from scratch. Instead, we usually start with a rough approximation of the desired behavior and take the learning from there. In this paper, we use imitation to quickly generate a rough solution to a robotic task from demonstrations, supplied as a collection of state-space trajectories. Appropriate control actions needed to steer the system along the trajectories are then automatically learned in the form of a (nonlinear) state-feedback control law. The learning scheme has two components: a dynamic reference model and an adaptive inverse process model, both based on a data-driven, non-parametric method called local linear regression. The reference model infers the desired behavior from the demonstration trajectories, while the inverse process model provides the control actions to achieve this behavior and is improved online using learning. Experimental results with a pendulum swing-up problem and a robotic arm demonstrate the practical usefulness of this approach. The resulting learned dynamics are not limited to single trajectories, but capture instead the overall dynamics of the motion, making the proposed approach a promising step towards versatile learning machines such as future household robots, or robots for autonomous missions.

Mots clés: machine learning; adaptive control; imitation learning; reference model

Domaines: Automatique / Robotique

Maarten Vaandrager, Robert Babuska, Lucian Busoniu, Gabriel Lopes. Imitation Learning with Non-Parametric Regression. IEEE International Conference on Automation Quality and Testing Robotics, AQTR 2012, May 2012, Cluj-Napoca, Romania. pp.91-96, ⟨10.1109/AQTR.2012.6237681⟩. ⟨hal-00756449⟩