Service interruption on Monday 11 July from 12:30 to 13:00: all the sites of the CCSD (HAL, EpiSciences, SciencesConf, AureHAL) will be inaccessible (network hardware connection).
Skip to Main content Skip to Navigation
Journal articles

DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning

Theo Jaunet 1 Romain Vuillemot 1 Christian Wolf 2 
1 SICAL - Situated Interaction, Collaboration, Adaptation and Learning
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
2 imagine - Extraction de Caractéristiques et Identification
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co-correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explainability in the field of visual analytics.
Complete list of metadata

Cited literature [61 references]  Display  Hide  Download
Contributor : Christian Wolf Connect in order to contact the contributor
Submitted on : Monday, October 19, 2020 - 11:16:47 AM
Last modification on : Monday, August 30, 2021 - 2:24:01 PM
Long-term archiving on: : Wednesday, January 20, 2021 - 6:30:32 PM


Files produced by the author(s)


  • HAL Id : hal-02864138, version 1
  • ARXIV : 1909.02982


Theo Jaunet, Romain Vuillemot, Christian Wolf. DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning. Computer graphics Forum (Proc. Eurovis), 2020. ⟨hal-02864138⟩



Record views


Files downloads