Skip to Main content Skip to Navigation
Conference papers

Learning of Binocular Fixations using Anomaly Detection with Deep Reinforcement Learning

Abstract : Due to its ability to learn complex behaviors in high-dimensional state-action spaces, deep reinforcement learning algorithms have attracted much interest in the robotics community. For a practical reinforcement learning implementation on a robot, it has to be provided with an informative reward signal that makes it easy to discriminate the values of nearby states. To address this issue, prior information, e.g. in the form of a geometric model, or human supervision are often assumed. This paper proposes a method to learn binocular fixations without such prior information. Instead, it uses an informative reward requiring little supervised information. The reward computation is based on an anomaly detection mechanism which uses convolutional autoencoders. These detectors estimate in a weakly supervised way an object's pixellic position. This position estimate is affected by noise, which makes the reward signal noisy. We first show that this affects both the learning speed and the resulting policy. Then, we propose a method to partially remove the noise using regression on the detection change given sensor data. The binocular fixation task is learned in a simulated environment on an object training set with various shapes and colors. The learned policy is compared with another one learned with a highly informative and noiseless reward signal. The tests are carried out on the training set and on a test set of new objects. We observe similar performances, showing that the environment-encoding step can replace the prior information.
Document type :
Conference papers
Complete list of metadata

Cited literature [19 references]  Display  Hide  Download
Contributor : François de la Bourdonnaye <>
Submitted on : Tuesday, January 30, 2018 - 9:20:30 AM
Last modification on : Wednesday, February 24, 2021 - 4:16:02 PM


Files produced by the author(s)



François de la Bourdonnaye, Céline Teulière, Thierry Chateau, Jochen Triesch. Learning of Binocular Fixations using Anomaly Detection with Deep Reinforcement Learning. International joint Conference on Neural Networks, May 2017, Anchorage, AK, United States. ⟨10.1109/IJCNN.2017.7965928⟩. ⟨hal-01635610v2⟩



Record views


Files downloads