W. Alexander and J. Brown, Computational Models of Performance Monitoring and Cognitive Control, Topics in Cognitive Science, vol.46, issue.4, pp.658-677, 2010.
DOI : 10.1111/j.1756-8765.2010.01085.x

W. Alexander and J. Brown, Medial prefrontal cortex as an action-outcome predictor, Nature Neuroscience, vol.8, issue.10, pp.1338-1344, 2011.
DOI : 10.1073/pnas.012470999
URL : http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3183374

C. Amiez, J. Joseph, and E. Procyk, Anterior cingulate error-related activity is modulated by predicted reward, European Journal of Neuroscience, vol.7, issue.12, pp.3447-3452, 2005.
DOI : 10.1111/j.1460-9568.2005.04170.x
URL : https://hal.archives-ouvertes.fr/inserm-00132130

C. Amiez, R. Neveu, D. Warrot, M. Petrides, K. Knoblauch et al., The Location of Feedback-Related Activity in the Midcingulate Cortex Is Predicted by Local Morphology, Journal of Neuroscience, vol.33, issue.5, pp.2217-2228, 2013.
DOI : 10.1523/JNEUROSCI.2779-12.2013

G. Aston-jones and J. Cohen, AN INTEGRATIVE THEORY OF LOCUS COERULEUS-NOREPINEPHRINE FUNCTION: Adaptive Gain and Optimal Performance, Annual Review of Neuroscience, vol.28, issue.1, pp.403-450, 2005.
DOI : 10.1146/annurev.neuro.28.061604.135709

R. Barkley, Linkages between attention and executive functions Attention, memory and executive function P, pp.307-326, 2001.

D. Barraclough, M. Conroy, and D. Lee, Prefrontal cortex and decision making in a mixed-strategy game, Nature Neuroscience, vol.7, issue.4, pp.404-410, 2004.
DOI : 10.1038/nn1209

F. Bartumeus, M. Da-luz, G. Viswanathan, and J. Catalan, ANIMAL SEARCH STRATEGIES: A QUANTITATIVE RANDOM-WALK ANALYSIS, Ecology, vol.86, issue.11, pp.3078-2087, 2005.
DOI : 10.1890/0012-9658(1999)080[1019:SSFLLI]2.0.CO;2

T. Behrens, L. Hunt, and M. Rushworth, The Computation of Social Behavior, Science, vol.324, issue.5931, pp.1160-1164, 2009.
DOI : 10.1126/science.1169694

T. Behrens, M. Woolrich, M. Walton, and M. Rushworth, Learning the value of information in an uncertain world, Nature Neuroscience, vol.1104, issue.9, pp.1214-1221, 2007.
DOI : 10.1038/nn1954

M. Botvinick, T. Braver, D. Barch, C. Carter, and J. Cohen, Conflict monitoring and cognitive control., Psychological Review, vol.108, issue.3, pp.624-652, 2001.
DOI : 10.1037/0033-295X.108.3.624

J. Brown and T. Braver, Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex, Science, vol.307, issue.5712, pp.1118-1121, 2005.
DOI : 10.1126/science.1105783

J. Cohen, G. Aston-jones, and M. Gilzenrat, A systems-level perspective on attention and cognitive control, Cognitive Neuroscience of attention, pp.71-90, 2004.

J. Cohen, S. Mcclure, and A. Yu, Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration, Philosophical Transactions of the Royal Society B: Biological Sciences, vol.46, issue.4, pp.933-942, 2007.
DOI : 10.1037/0033-295X.111.4.939

A. Collins and M. Frank, How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis, European Journal of Neuroscience, vol.22, issue.7, pp.1024-1035, 2012.
DOI : 10.1111/j.1460-9568.2011.07980.x

N. Daw, Trial-by-trial data analysis using computational models, 2011.
DOI : 10.1093/acprof:oso/9780199600434.003.0001

N. Daw, O. Doherty, J. Dayan, P. Seymour, B. Dolan et al., Cortical substrates for exploratory decisions in humans, Nature, vol.15, issue.7095, pp.876-879, 2006.
DOI : 10.1038/nature04766

D. Lillo, C. Visalerberghi, E. Aversano, and M. , The organization of exhaustive searches in a patchy space by capuchin monkeys (Cebus apella)., Journal of Comparative Psychology, vol.111, issue.1, 1997.
DOI : 10.1037/0735-7036.111.1.82

S. Dehaene, M. Kerszberg, and J. Changeux, A Neuronal Model of a Global Workspace in Effortful Cognitive Tasks, Annals of the New York Academy of Sciences, vol.18, issue.1, pp.14529-14534, 1998.
DOI : 10.1111/j.1749-6632.2001.tb05714.x

T. Desrochers, D. Jin, N. Goodman, and A. Graybiel, Optimal habits can develop spontaneously through sensitivity to local cost, Proceedings of the National Academy of Sciences, vol.107, issue.47, pp.20512-20517, 2010.
DOI : 10.1073/pnas.1013470107

P. Domenech and J. Dreher, Decision Threshold Modulation in the Human Brain, Journal of Neuroscience, vol.30, issue.43, pp.14305-14317, 2010.
DOI : 10.1523/JNEUROSCI.2371-10.2010
URL : https://hal.archives-ouvertes.fr/hal-00627818

N. Dosenbach, K. Visscher, E. Palmer, F. Miezin, K. Wenger et al., A Core System for the Implementation of Task Sets, Neuron, vol.50, issue.5, pp.799-812, 2006.
DOI : 10.1016/j.neuron.2006.04.031

K. Doya, Metalearning and neuromodulation, Neural Networks, vol.15, issue.4-6, pp.495-506, 2002.
DOI : 10.1016/S0893-6080(02)00044-8

D. Durstewitz and J. Seamans, The Dual-State Theory of Prefrontal Cortex Dopamine Function with Relevance to Catechol-O-Methyltransferase Genotypes and Schizophrenia, Biological Psychiatry, vol.64, issue.9, pp.739-749, 2008.
DOI : 10.1016/j.biopsych.2008.05.015

D. Durstewitz, N. Vittoz, S. Floresco, and J. Seamans, Abrupt Transitions between Prefrontal Neural Ensemble States Accompany Behavioral Transitions during Rule Learning, Neuron, vol.66, issue.3, pp.438-448, 2010.
DOI : 10.1016/j.neuron.2010.03.029

. Khamassi, Adaptive control in prefrontal cortex 35

K. Enomoto, N. Matsumoto, S. Nakai, T. Satoh, T. Sato et al., Dopamine neurons learn to encode the long-term value of multiple future rewards, Proceedings of the National Academy of Sciences, vol.108, issue.37, pp.15462-15467, 2011.
DOI : 10.1073/pnas.1014457108

E. Fonio, Y. Benjamini, and I. Golani, Freedom of movement and the stability of its unfolding in free exploration of mice, Proceedings of the National Academy of Sciences, vol.106, issue.50, pp.21335-21340, 2009.
DOI : 10.1073/pnas.0812513106

M. Frank, B. Doll, J. Oas-terpstra, and F. Moreno, Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation, Nature Neuroscience, vol.23, issue.8, pp.1062-1068, 2009.
DOI : 10.1093/nar/29.17.e88

C. Holroyd and M. Coles, The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity., Psychological Review, vol.109, issue.4, pp.679-709, 2002.
DOI : 10.1037/0033-295X.109.4.679

M. Humphries, M. Khamassi, and K. Gurney, Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia, Frontiers in Neuroscience, vol.6, p.9, 2012.
DOI : 10.3389/fnins.2012.00009
URL : https://hal.archives-ouvertes.fr/hal-00688928

S. Ishii, W. Yoshida, and J. Yoshimoto, Control of exploitation???exploration meta-parameter in reinforcement learning, Neural Networks, vol.15, issue.4-6, pp.665-687, 2002.
DOI : 10.1016/S0893-6080(02)00056-4

M. Ito and K. Doya, Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia, Journal of Neuroscience, vol.29, issue.31, pp.9861-9874, 2009.
DOI : 10.1523/JNEUROSCI.6157-08.2009

D. Kaping, M. Vinck, R. Hutchison, S. Everling, and T. Womelsdorf, Specific Contributions of Ventromedial, Anterior Cingulate, and Lateral Prefrontal Cortex for Attentional Selection and Stimulus Valuation, PLoS Biology, vol.66, issue.12, p.1001224, 2011.
DOI : 10.1371/journal.pbio.1001224.s007

S. Kennerley and J. Wallis, Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables, European Journal of Neuroscience, vol.3, issue.10, pp.2061-2073, 2009.
DOI : 10.1111/j.1460-9568.2009.06743.x

S. Kennerley and M. Walton, Decision making and reward in frontal cortex: Complementary evidence from neurophysiological and neuropsychological studies., Behavioral Neuroscience, vol.125, issue.3, pp.297-317, 2011.
DOI : 10.1037/a0023575

S. Kennerley, M. Walton, T. Behrens, M. Buckley, and M. Rushworth, Optimal decision making and the anterior cingulate cortex, Nature Neuroscience, vol.336, issue.7, pp.940-947, 2006.
DOI : 10.1038/nn1724

J. Kerns, J. Cohen, A. Macdonald, R. Cho, V. Stenger et al., Anterior Cingulate Conflict Monitoring and Adjustments in Control, Science, vol.303, issue.5660, pp.1023-1026, 2004.
DOI : 10.1126/science.1089910

M. Khamassi, P. Enel, P. Dominey, and E. Procyk, Medial prefrontal cortex and the adaptive regulation of reinforcement learning parameters, Prog Brain Res, vol.202, pp.441-464, 2013.
DOI : 10.1016/B978-0-444-62604-2.00022-8

M. Khamassi, S. Lallee, P. Enel, E. Procyk, and P. Dominey, Robot Cognitive Control with a Neurophysiologically Inspired Reinforcement Learning Model, Frontiers in Neurorobotics, vol.5, issue.1, 2011.
DOI : 10.3389/fnbot.2011.00001
URL : https://hal.archives-ouvertes.fr/hal-00688931

N. Kolling, T. Behrens, R. Mars, and M. Rushworth, Neural Mechanisms of Foraging, Science, vol.336, issue.6077, pp.95-98, 2012.
DOI : 10.1126/science.1216930

F. Kouneiher, S. Charron, and E. Koechlin, Motivation and cognitive control in the human prefrontal cortex, Nature Neuroscience, vol.19, issue.7, pp.939-945, 2009.
DOI : 10.1016/j.neuroimage.2004.07.041

J. Krichmar, The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World, Adaptive Behavior, vol.46, issue.6, pp.385-399, 2008.
DOI : 10.1177/1059712308095775

C. Landmann, S. Dehaene, S. Pappata, A. Jobert, M. Bottlaender et al., Dynamics of Prefrontal and Cingulate Activity during a Reward-Based Logical Deduction Task, Cerebral Cortex, vol.17, issue.4, pp.749-759, 2007.
DOI : 10.1093/cercor/bhk028
URL : https://hal.archives-ouvertes.fr/hal-00349660

B. Lau, Matlab code for diagnosing collinearity in a regression design matrix. figshare, 2014.

H. Leung, J. Gore, and P. Goldman-rakic, Sustained Mnemonic Response in the Human Middle Frontal Gyrus during On-Line Storage of Spatial Memoranda, Journal of Cognitive Neuroscience, vol.75, issue.4, pp.659-671, 2002.
DOI : 10.1016/S0926-6410(99)00033-6

C. Luk and J. Wallis, Dynamic Encoding of Responses and Outcomes by Neurons in Medial Prefrontal Cortex, Journal of Neuroscience, vol.29, issue.23, pp.7526-7539, 2009.
DOI : 10.1523/JNEUROSCI.0386-09.2009

A. Macdonald, J. Cohen, V. Stenger, and C. Carter, Dissociating the Role of the Dorsolateral Prefrontal and Anterior Cingulate Cortex in Cognitive Control, Science, vol.288, issue.5472, pp.1835-1838, 2000.
DOI : 10.1126/science.288.5472.1835

M. Matsumoto, K. Matsumoto, H. Abe, and K. Tanaka, Medial prefrontal cell activity signaling prediction errors of action values, Nature Neuroscience, vol.93, issue.5, pp.647-656, 2007.
DOI : 10.1126/science.1069504

S. Mcclure, M. Gilzenrat, and J. Cohen, An exploration?exploitation model based on norepinephrine and dopamine activity Advances in neural information processing systems, pp.867-874, 2006.

S. Panzeri, R. Senatore, M. Montemurro, and R. Petersen, Correcting for the Sampling Bias Problem in Spike Train Information Measures, Journal of Neurophysiology, vol.98, issue.3, pp.1064-1072, 2007.
DOI : 10.1152/jn.00559.2007

S. Panzeri and A. Treves, Analytical estimates of limited sampling biases in different information measures, Network: Computation in Neural Systems, vol.7, issue.1, pp.87-107, 1996.
DOI : 10.1103/PhysRevE.52.6841

E. Procyk and P. Goldman-rakic, Modulation of Dorsolateral Prefrontal Delay Activity during Self-Organized Behavior, Journal of Neuroscience, vol.26, issue.44, pp.11313-11323, 2006.
DOI : 10.1523/JNEUROSCI.2157-06.2006
URL : https://hal.archives-ouvertes.fr/inserm-00132158

. Khamassi, Adaptive control in prefrontal cortex 36

E. Procyk, Y. Tanaka, and J. Joseph, Anterior cingulate activity during routine and non-routine sequential behaviors in macaques, Nature Neuroscience, vol.3, issue.5, pp.502-508, 2000.
DOI : 10.1038/74880
URL : https://hal.archives-ouvertes.fr/inserm-00132133

Q. Quiroga, R. Panzeri, and S. , Extracting information from neuronal populations: information theory and decoding approaches, Nature Reviews Neuroscience, vol.335, issue.3, pp.173-185, 2009.
DOI : 10.1523/JNEUROSCI.5319-04.2005

R. Quilodran, M. Rothé, and E. Procyk, Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex, Neuron, vol.57, issue.2, pp.314-325, 2008.
DOI : 10.1016/j.neuron.2007.11.031
URL : https://hal.archives-ouvertes.fr/inserm-00906686

M. Rothe, R. Quilodran, J. Sallet, and E. Procyk, Coordination of High Gamma Activity in Anterior Cingulate and Lateral Prefrontal Cortical Areas during Adaptation, Journal of Neuroscience, vol.31, issue.31, pp.11110-11117, 2011.
DOI : 10.1523/JNEUROSCI.1016-11.2011

M. Rushworth and T. Behrens, Choice, uncertainty and value in prefrontal and cingulate cortex, Nature Neuroscience, vol.9, issue.4, pp.389-397, 2008.
DOI : 10.1038/nn2066

T. Satoh, S. Nakai, T. Sato, and M. Kimura, Correlated coding of motivation and outcome of decision by dopamine neurons, J Neurosci, vol.23, pp.9913-9923, 2003.

W. Schultz, P. Dayan, and P. Montague, A Neural Substrate of Prediction and Reward, Science, vol.275, issue.5306, pp.1593-1599, 1997.
DOI : 10.1126/science.275.5306.1593

N. Schweighofer and K. Doya, Meta-learning in Reinforcement Learning, Neural Networks, vol.16, issue.1, pp.5-9, 2003.
DOI : 10.1016/S0893-6080(02)00228-9

H. Seo and D. Lee, Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game, Journal of Neuroscience, vol.27, issue.31, pp.8366-8377, 2007.
DOI : 10.1523/JNEUROSCI.2369-07.2007

H. Seo and D. Lee, Cortical mechanisms for reinforcement learning in competitive games, Philosophical Transactions of the Royal Society B: Biological Sciences, vol.447, issue.7148, pp.3845-3857, 2008.
DOI : 10.1038/nature05852

H. Seo and D. Lee, Behavioral and Neural Changes after Gains and Losses of Conditioned Reinforcers, Journal of Neuroscience, vol.29, issue.11, pp.3627-3641, 2009.
DOI : 10.1523/JNEUROSCI.4726-08.2009

R. Sutton and A. Barto, Reinforcement Learning: An Introduction, IEEE Transactions on Neural Networks, vol.9, issue.5, 1998.
DOI : 10.1109/TNN.1998.712192

A. Treves and S. Panzeri, The Upward Bias in Measures of Information Derived from Limited Data Samples, Neural Computation, vol.17, issue.2, pp.399-407, 1995.
DOI : 10.1007/BF00962721

B. Vogt, L. Vogt, N. Farber, and G. Bush, Architecture and neurocytology of monkey cingulate gyrus, The Journal of Comparative Neurology, vol.70, issue.3, pp.218-239, 2005.
DOI : 10.1002/cne.20512

X. Wang, Neurophysiological and Computational Principles of Cortical Rhythms in Cognition, Physiological Reviews, vol.90, issue.3, pp.1195-1268, 2010.
DOI : 10.1152/physrev.00035.2008

C. Wilson, D. Gaffan, P. Browning, and M. Baxter, Functional localization within the prefrontal cortex: missing the forest for the trees? Trends Neurosci, pp.533-540, 2010.