Why and How Can Hippocampal Transition Cells Be Used in Reinforcement Learning ?

Julien Hirel (1) , Philippe Gaussier (1) , Mathias Quoy (1) , Jean-Paul Banquet (1)

1 ETIS - UMR 8051 - Equipes Traitement de l'Information et Systèmes

In this paper we present a model of reinforcement learning (RL) which can be used to solve goal-oriented navigation tasks. Our model supposes that transitions between places are learned in the hip- pocampus (CA pyramidal cells) and associated with information coming from path-integration. The RL neural network acts as a bias on these transitions to perform action selection. RL originates in the basal ganglia and matches observations of reward-based activity in dopaminergic neu- rons. Experiments were conducted in a simulated environment. We show that our model using transitions and inspired by Q-learning performs more efficiently than traditional actor-critic models of the basal ganglia based on temporal difference (TD) learning and using static states.

Domaines: Réseau de neurones [cs.NE]; Robotique [cs.RO]; Apprentissage [cs.LG]

Julien Hirel, Philippe Gaussier, Mathias Quoy, Jean-Paul Banquet. Why and How Can Hippocampal Transition Cells Be Used in Reinforcement Learning ?. SAB, Aug 2010, Paris, France. pp.359-369, ⟨10.1007/978-3-642-15193-4_3⟩. ⟨hal-00520113⟩