Dissociation of reinforcement and Hebbian learning induces covert acquisition of value in the basal ganglia

Meropi Topalidou (1, 2, 3, 4) , Daisuke Kase (2, 4) , Thomas Boraud (2, 4) , Nicolas P. Rougier (1, 2, 3, 4)

1 Mnemosyne - Mnemonic Synergy
2 IMN - Institut des Maladies Neurodégénératives [Bordeaux]
3 LaBRI - Laboratoire Bordelais de Recherche en Informatique
4 UB - Université de Bordeaux

This paper introduces a new hypothesis concerning the formation of habits in the cortex of primates under the implicit supervision of the basal ganglia. This hypothesis has been formulated using a theoretical model and confirmed experimentally in monkeys. To do so, and prior to learning, we inactivated the internal part of the globus pallidus (GPi, the main output structure of the BG) with injections of muscimol and we tested monkeys on a variant of a two-armed bandit task where two stimuli are associated with two distinct reward probabilities (0.25 and 0.75 respectively). Unsurprisingly, their performance in such conditions are at the chance level. However, the theoretical model predicts that even if the performance is random, the value of the stimuli are implicitly evaluated and learned. This has been tested and confirmed on the next day, when inhibition has been removed: monkeys instantly showed quasi-optimal performances, demonstrating they knew the relative value of the two stimuli. Said differently, we managed to explicitly dissociate reinforcement learning from Hebbian learning and demonstrated covert learning inside the basal ganglia. These results suggest that a behavioral decision results from both the cooperation (acquisition) and competition (expression) of two distinct but entangled memory systems, the goal-directed system and the habit system that may represent the two ends of the same graded phenomenon.

Mots clés: habits; action selection; cortex; basal ganglia; computational model; value acquisition; habit expression; covert learning; reinforcement learning; Hebbian learning

Domaines: Neurosciences; Sciences cognitives

Meropi Topalidou, Daisuke Kase, Thomas Boraud, Nicolas P. Rougier. Dissociation of reinforcement and Hebbian learning induces covert acquisition of value in the basal ganglia. 2016. ⟨hal-01337332v1⟩