Gain-Field Modulation Mechanism in Multimodal Networks for Spatial Perception

Alexandre Pitti (1) , Arnaud Blanchard (1) , Matthieu Cardinaux (1) , Philippe Gaussier (2)

1 Neurocybernétique
2 Neurocybernetique

Seeing is not just done through the eyes, it involves the integration of other modalities such as auditory, proprio- ceptive and tactile information, to locate objets, persons and also the limbs. We hypothesize that the neural mechanism of gain-field modulation, which is found to process coordinate transform between modalities in the superior colliculus and in the parietal area, plays a key role to build such unified perceptual world. In experiments with a head-neck-eye's robot with a camera and microphones, we study how gain-field modulation in neural networks can serve for transcribing one modality's reference frame into another one (e.g., audio signals into eyes' coordinate). It follows that each modality influences the estimations of the position of a stimulus (multimodal enhancement). This can be used in example for mapping sound signals into retina coordinates for audio-visual speech perception.

Domaines: Robotique [cs.RO]

Alexandre Pitti, Arnaud Blanchard, Matthieu Cardinaux, Philippe Gaussier. Gain-Field Modulation Mechanism in Multimodal Networks for Spatial Perception. 2012 12th IEEE-RAS International Conference on Humanoid Robots Nov.29-Dec.1, 2012. Business Innovation Center Osaka, Japan, Nov 2012, Osaka, Japan. pp.297-302. ⟨hal-00762739⟩