Crossmodal Adaptation in Right Posterior Superior Temporal Sulcus during Face-Voice Emotional Integration

Rebecca Watson (1, 2) , Marianne Latinus (2, 3) , Takao Noguchi (4) , Oliver Garrod (2) , Frances Crabbe (2) , Pascal Belin (2, 3, 5)

1 Maastricht University [Maastricht]
2 Institute of Neurosciences and Psychology [Glasgow]
3 INT - Institut de Neurosciences de la Timone
4 University of Warwick [Coventry]
5 BRAMS - International Laboratory for Brain, Music and Sound Research

The integration of emotional information from the face and voice of other persons is known to be mediated by a number of "multisen-sory" cerebral regions, such as the right posterior superior temporal sulcus (pSTS). However, whether multimodal integration in these regions is attributable to interleaved populations of unisensory neurons responding to face or voice or rather by multimodal neurons receiving input from the two modalities is not fully clear. Here, we examine this question using functional magnetic resonance adaptation and dynamic audiovisual stimuli in which emotional information was manipulated parametrically and independently in the face and voice via morphing between angry and happy expressions. Healthy human adult subjects were scanned while performing a happy/angry emotion categorization task on a series of such stimuli included in a fast event-related, continuous carryover design. Subjects integrated both face and voice information when categorizing emotion-although there was a greater weighting of face information-and showed behavioral adaptation effects both within and across modality. Adaptation also occurred at the neural level: in addition to modality-specific adaptation in visual and auditory cortices, we observed for the first time a crossmodal adaptation effect. Specifically, fMRI signal in the right pSTS was reduced in response to a stimulus in which facial emotion was similar to the vocal emotion of the preceding stimulus. These results suggest that the integration of emotional information from face and voice in the pSTS involves a detectable proportion of bimodal neurons that combine inputs from visual and auditory cortices.

Mots clés: emotion perception; functional magnetic resonance adaptation; multisensory integration

Domaines: Sciences cognitives; Neurobiologie

Rebecca Watson, Marianne Latinus, Takao Noguchi, Oliver Garrod, Frances Crabbe, et al.. Crossmodal Adaptation in Right Posterior Superior Temporal Sulcus during Face-Voice Emotional Integration. Journal of Neuroscience, 2014, 34 (20), pp.6813-6821. ⟨10.1523/jneurosci.4478-13.2014⟩. ⟨hal-02005297⟩