Skip to Main content Skip to Navigation
Conference papers

EgoMap: Projective mapping and structured egocentric memory for Deep RL

Edward Beeching 1 Jilles Dibangoye 1 Olivier Simonin 1 Christian Wolf 2
1 CHROMA - Robots coopératifs et adaptés à la présence humaine en environnements dynamiques
Inria Grenoble - Rhône-Alpes, CITI - CITI Centre of Innovation in Telecommunications and Integration of services
2 imagine - Extraction de Caractéristiques et Identification
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : Tasks involving localization, memorization and planning in partially observable 3D environments are an ongoing challenge in Deep Reinforcement Learning. We present EgoMap, a spatially structured neural memory architecture. EgoMap augments a deep reinforcement learning agent's performance in 3D environments on challenging tasks with multi-step objectives. The EgoMap architecture incorporates several inductive biases including a differentiable inverse projection of CNN feature vectors onto a top-down spatially structured map. The map is updated with ego-motion measurements through a differentiable affine transform. We show this architecture outperforms both standard recurrent agents and state of the art agents with structured memory. We demonstrate that incorporating these inductive biases into an agent's architecture allows for stable training with reward alone, circumventing the expense of acquiring and labelling expert trajectories. A detailed ablation study demonstrates the impact of key aspects of the architecture and through extensive qualitative analysis, we show how the agent exploits its structured internal memory to achieve higher performance.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02864146
Contributor : Christian Wolf <>
Submitted on : Wednesday, June 10, 2020 - 9:34:07 PM
Last modification on : Wednesday, July 8, 2020 - 12:43:53 PM

Links full text

Identifiers

  • HAL Id : hal-02864146, version 1
  • ARXIV : 2002.02286

Citation

Edward Beeching, Jilles Dibangoye, Olivier Simonin, Christian Wolf. EgoMap: Projective mapping and structured egocentric memory for Deep RL. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), Sep 2020, ghent, Belgium. ⟨hal-02864146⟩

Share

Metrics

Record views

17