Inside Speech: Multisensory and Modality-specific Processing of Tongue and Lip Speech Actions

Abstract : Action recognition has been found to rely not only on sensory brain areas but also partly on the observer's motor system. However, whether distinct auditory and visual experiences of an action modulate sensorimotor activity remains largely unknown. In the present sparse sampling fMRI study, we determined to which extent sensory and motor representations interact during the perception of tongue and lip speech actions. Tongue and lip speech actions were selected because tongue movements of our interlocutor are accessible via their impact on speech acoustics but not visible because of its position inside the vocal tract, whereas lip movements are both " audible " and visible. Participants were presented with auditory, visual, and audiovisual speech actions, with the visual inputs related to either a sagittal view of the tongue movements or a facial view of the lip movements of a speaker, previously recorded by an ultrasound imaging system and a video camera. Although the neural networks involved in visual visuo-lingual and visuo-facial perception largely overlapped, stronger motor and somato-sensory activations were observed during visuo-lingual perception. In contrast, stronger activity was found in auditory and visual cortices during visuo-facial perception. Complementing these findings, activity in the left premotor cortex and in visual brain areas was found to correlate with visual recognition scores observed for visuo-lingual and visuo-facial speech stimuli , respectively, whereas visual activity correlated with RTs for both stimuli. These results suggest that unimodal and multi-modal processing of lip and tongue speech actions rely on common sensorimotor brain areas. They also suggest that visual processing of audible but not visible movements induces motor and visual mental simulation of the perceived actions to facilitate recognition and/or to learn the association between auditory and visual signals.
Type de document :
Article dans une revue
Journal of Cognitive Neuroscience, Massachusetts Institute of Technology Press (MIT Press), 2017, 29, pp.448 - 466. 〈10.1162/jocn_a_01057〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01484978
Contributeur : Avril Treille <>
Soumis le : mercredi 8 mars 2017 - 09:36:43
Dernière modification le : jeudi 16 mars 2017 - 16:15:06
Document(s) archivé(s) le : vendredi 9 juin 2017 - 12:37:51

Fichier

JOCN_a_01057-Treille_Proof1_co...
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Avril Treille, Coriandre Vilain, Thomas Hueber, Laurent Lamalle, Marc Sato. Inside Speech: Multisensory and Modality-specific Processing of Tongue and Lip Speech Actions. Journal of Cognitive Neuroscience, Massachusetts Institute of Technology Press (MIT Press), 2017, 29, pp.448 - 466. 〈10.1162/jocn_a_01057〉. 〈hal-01484978〉

Partager

Métriques

Consultations de
la notice

414

Téléchargements du document

58