Hippocampal replays under the scrutiny of reinforcement learning models

Romain Cazé (1) , Mehdi Khamassi (1) , Lise Aubin (1) , Benoît Girard (1, 2)

1 ISIR - Institut des Systèmes Intelligents et de Robotique
2 AMAC - Architectures et modèles d'Adptation et de la cognition

Multiple in vivo studies have shown that place cells from the hippocampus replay previously experienced trajectories. These replays are commonly considered to mainly reflect memory consolidation processes. Some data, however , have highlighted a functional link between replays and reinforcement learning (RL). This theory, extensively used in machine learning, has introduced efficient algorithms and can explain various behavioral and physiological measures from different brain regions. RL algorithms could constitute a mechanistic description of replays, and explain how replays can reduce the number of iterations required to explore the environment during learning. We review here the main findings concerning the different hippocampal replay types and the possible associated RL models (either model-based, model-free or hybrid model types). We conclude by tying these frameworks together. We illustrate the link between data and RL through a series of model simulations. This review, at the frontier between informatics and biology, paves the way for future work on replays.

Mots clés: Place cells; Hippocampus; Activity replay; Sleep; Computational modeling; Reinforcement learning; Model-free / Model-based

Domaines: Neurosciences [q-bio.NC]

Romain Cazé, Mehdi Khamassi, Lise Aubin, Benoît Girard. Hippocampal replays under the scrutiny of reinforcement learning models. Journal of Neurophysiology, 2018, ⟨10.1152/jn.00145.2018⟩. ⟨hal-02323528⟩