Skip to Main content Skip to Navigation

Reinforcement Learning Using Neural Networks, with Applications to Motor Control

Abstract : This thesis is a study of practical methods to estimate value functions with feedforward neural networks in model-based reinforcement learning. Focus is placed on problems in continuous time and space, such as motor-control tasks. In this work, the continuous TD(lambda) algorithm is refined to handle situations with discontinuous states and controls, and the vario-eta algorithm is proposed as a simple but efficient method to perform gradient descent. The main contributions of this thesis are experimental successes that clearly indicate the potential of feedforward neural networks to estimate high-dimensional value functions. Linear function approximators have been often preferred in reinforcement learning, but successful value function estimations in previous works are restricted to mechanical systems with very few degrees of freedom. The method presented in this thesis was tested successfully on an original task of learning to swim by a simulated articulated robot, with 4 control variables and 12 independent state variables, which is significantly more complex than problems that have been solved with linear function approximators so far.
Document type :
Complete list of metadata

Cited literature [79 references]  Display  Hide  Download
Contributor : Rémi Coulom Connect in order to contact the contributor
Submitted on : Tuesday, December 16, 2003 - 2:19:39 PM
Last modification on : Friday, November 6, 2020 - 4:13:00 AM
Long-term archiving on: : Friday, April 2, 2010 - 7:28:55 PM


  • HAL Id : tel-00003985, version 1




Rémi Coulom. Reinforcement Learning Using Neural Networks, with Applications to Motor Control. Other [cs.OH]. Institut National Polytechnique de Grenoble - INPG, 2002. English. ⟨tel-00003985⟩



Record views


Files downloads