Role of the frontal cortex in solving the exploration-exploitation trade-off

Mehdi Khamassi (1) , René Quilodran (1) , Pierre Enel (1) , Peter Dominey (1) , Emmanuel Procyk (1)

1 SBRI - Institut cellule souche et cerveau

While many electrophysiological recordings and computational modeling work have investigated the role of the frontal cortex in reinforcement learning (learning by trial-and-error to adapt action values), it is not yet clear how the brain flexibly regulates in a task-appropriate way crucial parameters of learning such as the learning rate and the exploration rate. In a previous work, we proposed a computational model based on the meta-learning theoretical framework where the frontal cortex extracts feedback signals 1) to update action values based on a reward prediction error, 2) to estimate the level of exploration based on the current reward average, 3) to select action based on this exploration rate. This model helped us draw a set of experimental predictions. Here we show a model-based analysis of single-unit recordings in the monkey prefrontal cortex so as to test these predictions. We found neural subpopulations activities consistent with these three functions. We also found global properties of the recorded neural ensemble - such as variations in spatial selectivity - which were predicted by our model. Such an approach, gathering computational modeling and neurophysiology, can help understand complex activities of neural ensembles related to decision making.

Mots clés: Action selection; Connectionnist models; System dynamics

Domaines: Neurosciences [q-bio.NC]; Neurosciences

Mehdi Khamassi, René Quilodran, Pierre Enel, Peter Dominey, Emmanuel Procyk. Role of the frontal cortex in solving the exploration-exploitation trade-off. Cinquième conférence plénière française de Neurosciences Computationnelles, "Neurocomp'10", Aug 2010, Lyon, France. ⟨hal-00553443⟩