Learning of Binocular Fixations using Anomaly Detection with Deep Reinforcement Learning

Abstract : Due to its ability to learn complex behaviors in high-dimensional state-action spaces, deep reinforcement learning algorithms have attracted much interest in the robotics community. For a practical reinforcement learning implementation on a robot, it has to be provided with an informative reward signal that makes it easy to discriminate the values of nearby states. To address this issue, prior information, e.g. in the form of a geometric model, or human supervision are often assumed. This paper proposes a method to learn binocular fixations without such prior information. Instead, it uses an informative reward requiring little supervised information. The reward computation is based on an anomaly detection mechanism which uses convolutional autoencoders. These detectors estimate in a weakly supervised way an object's pixellic position. This position estimate is affected by noise, which makes the reward signal noisy. We first show that this affects both the learning speed and the resulting policy. Then, we propose a method to partially remove the noise using regression on the detection change given sensor data. The binocular fixation task is learned in a simulated environment on an object training set with various shapes and colors. The learned policy is compared with another one learned with a highly informative and noiseless reward signal. The tests are carried out on the training set and on a test set of new objects. We observe similar performances, showing that the environment-encoding step can replace the prior information.
Type de document :
Communication dans un congrès
International joint Conference on Neural Networks, May 2017, Anchorage, AK, United States. IEEE, 2017 International Joint Conference on Neural Networks (IJCNN), 〈10.1109/IJCNN.2017.7965928〉
Liste complète des métadonnées

Littérature citée [19 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01635610
Contributeur : François De La Bourdonnaye <>
Soumis le : mardi 30 janvier 2018 - 09:20:30
Dernière modification le : vendredi 9 février 2018 - 01:26:23

Fichier

fdlb5.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

François De La Bourdonnaye, Céline Teulière, Thierry Chateau, Jochen Triesch. Learning of Binocular Fixations using Anomaly Detection with Deep Reinforcement Learning. International joint Conference on Neural Networks, May 2017, Anchorage, AK, United States. IEEE, 2017 International Joint Conference on Neural Networks (IJCNN), 〈10.1109/IJCNN.2017.7965928〉. 〈hal-01635610v2〉

Partager

Métriques

Consultations de la notice

95

Téléchargements de fichiers

37