Skip to Main content Skip to Navigation
Journal articles

Robot cognitive control with a neurophysiologically inspired reinforcement learning model

Abstract : A major challenge in modern robotics is to liberate robots from controlled industrial settings, and allow them to interact with humans and changing environments in the real world. The current research attempts to determine if a neurophysiologically motivated model of cortical function in the primate can help to address this challenge. Primates are endowed with cognitive systems that allow them to maximize the feedback from their environment by learning the values of actions in diverse situations and by adjusting their behavioral parameters (i.e. cognitive control) to accommodate unexpected events. In such contexts uncertainty can arise from at least two distinct sources - expected uncertainty resulting from noise during sensory-motor interaction in a known context, and unexpected uncertainty resulting from the changing probabilistic structure of the environment. However, it is not clear how neurophysiological mechanisms of reinforcement learning and cognitive control integrate in the brain to produce efficient behavior. Based on primate neuroanatomy and neurophysiology, we propose a novel computational model for the interaction between lateral prefrontal and anterior cingulate cortex (LPFC and ACC) reconciling previous models dedicated to these two functions. We deployed the model in two robots and demonstrate that, based on adaptive regulation of a meta-parameter β that controls the exploration rate, the model can robustly deal with the two kinds of uncertainties in the real world. In addition the model could reproduce monkey behavioral performance and neurophysiological data in two problem-solving tasks. A last experiment extends this to human-robot interaction with the iCub humanoid, and novel sources of uncertainty corresponding to "cheating" by the human. The combined results provide concrete evidence for the ability of neurophysiologically inspired cognitive systems to control advanced robots in the real world.
Complete list of metadata

Cited literature [45 references]  Display  Hide  Download
Contributor : Mehdi Khamassi Connect in order to contact the contributor
Submitted on : Wednesday, April 18, 2012 - 9:42:54 PM
Last modification on : Sunday, June 26, 2022 - 5:39:23 AM
Long-term archiving on: : Thursday, July 19, 2012 - 2:32:31 AM


Publisher files allowed on an open archive




Mehdi Khamassi, Stéphane Lallée, Pierre Enel, Emmanuel Procyk, Peter Dominey. Robot cognitive control with a neurophysiologically inspired reinforcement learning model. Frontiers in Neurorobotics, Frontiers, 2011, 5 (1), pp.1-13. ⟨10.3389/fnbot.2011.00001⟩. ⟨hal-00688931⟩



Record views


Files downloads