Touching your lips, hearing your tongue, seeing your voice
Résumé
Do cross-modal interactions during speech perception only depend on well-known auditory and visuo-facial modalities or might also be triggered by other sensory sources less common in speech communication? In a first set of EEG studies, auditory, audio-visual and audio-haptic speech perception were compared through natural dyadic interactions between a listener and a speaker. In the haptic modality, participants were presented with syllables felt from manual tactile contact with the speaker's face. In a second set of EEG and fMRI studies, cross-modal binding between auditory, visuo-facial and visuo-lingual speech signals was investigated. The visuo-facial and visuo-lingual modalities consisted on either a facial view of the lip movements or a sagittal view of the tongue movements of a speaker, recorded by a camera and an ultrasound imaging system. A third set of EEG and fMRI studies examined the impact of self-knowledge during the perception of auditory, visual and audio-visual speech stimuli that were previously recorded by a participant or a speaker he/she had never met. Altogether our results provide new evidence for cross-modal interactions between auditory, haptic, visuo-facial and visuo-lingual speech signals and demonstrate a processing advantage when the perceptual situation involves our own speech production.