Skip to Main content Skip to Navigation
Conference papers

Learning of Binocular Fixations using Anomaly Detection with Deep Reinforcement Learning

Abstract : Due to its ability to learn complex behaviors in high-dimensional state-action spaces, deep reinforcement learning algorithms have attracted much interest in the robotics community. For a practical reinforcement learning implementation on a robot, it has to be provided with an informative reward signal that makes it easy to discriminate the values of nearby states. To address this issue, prior information, e.g. in the form of a geometric model, or human supervision are often assumed. This paper proposes a method to learn binocular fixations without such prior information. Instead, it uses an informative reward requiring little supervised information. The reward computation is based on an anomaly detection mechanism which uses convolutional autoencoders. These detectors estimate in a weakly supervised way an object's pixellic position. This position estimate is affected by noise, which makes the reward signal noisy. We first show that this affects both the learning speed and the resulting policy. Then, we propose a method to partially remove the noise using regression on the detection change given sensor data. The binocular fixation task is learned in a simulated environment on an object training set with various shapes and colors. The learned policy is compared with another one learned with a highly informative and noiseless reward signal. The tests are carried out on the training set and on a test set of new objects. We observe similar performances, showing that the environment-encoding step can replace the prior information.
Document type :
Conference papers
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01635610
Contributor : François de la Bourdonnaye <>
Submitted on : Tuesday, January 30, 2018 - 9:20:30 AM
Last modification on : Friday, February 9, 2018 - 1:26:23 AM

File

fdlb5.pdf
Files produced by the author(s)

Identifiers

Citation

François de la Bourdonnaye, Céline Teulière, Thierry Chateau, Jochen Triesch. Learning of Binocular Fixations using Anomaly Detection with Deep Reinforcement Learning. International joint Conference on Neural Networks, May 2017, Anchorage, AK, United States. ⟨10.1109/IJCNN.2017.7965928⟩. ⟨hal-01635610v2⟩

Share

Metrics

Record views

231

Files downloads

689