Skip to Main content Skip to Navigation
Conference papers

Now you see me: finding the right observation space to learn diverse behaviours by reinforcement in games

Abstract : Training virtual agents to play a game using reinforcement learning (RL) has gained a lot of traction in recent years. Indeed, RL has delivered agents with superhuman performances on multiple gameplays. Yet, from a human-machine interaction standpoint, raw performance is not the only dimension of a "good" game AI. Exhibiting diversified behaviours is key to generate novelty, one of the core components of player engagement. In the RL framework, teaching agents to discover multiple strategies to achieve the same task is often framed as skill discovery. However, we observe that the current RL literature defines diversity as the exploration of different states, i.e. the incentive of the agent to "see" new observations. In this work, we argue that this definition does not make sense from a gameplay point of view. Instead, diversity should be defined as a distance on observations from an observer, external to the agent. We illustrate how DIAYN/SMERL, state of the art RL algorithms for skill discovery, fail to discover meaningful behaviours in a simple tag game. We propose an easy fix by introducing the notion of diversity spaces, defined as the observations gathered by a third-party external to the agent.
Complete list of metadata
Contributor : Nicolas Audebert Connect in order to contact the contributor
Submitted on : Wednesday, May 25, 2022 - 11:45:13 AM
Last modification on : Friday, August 5, 2022 - 2:54:00 PM


Files produced by the author(s)


  • HAL Id : hal-03678280, version 1



Raphaël Boige, Nicolas Audebert, Clément Rambour, Guillaume Levieux. Now you see me: finding the right observation space to learn diverse behaviours by reinforcement in games. Conférence sur l'Apprentissage automatique (CAp), Jul 2022, Vannes, France. ⟨hal-03678280⟩



Record views


Files downloads