How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Brain and Language Année : 2022

How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing

Résumé

Articulatory gestures and orthography are connected with speech through a natural and an artificial association, respectively. This EEG study investigated whether the integrations between speech and these two visual inputs rely on the same mechanism, despite their different characteristics. A comparison of skilled readers' brain responses elicited by spoken words presented alone versus synchronously with visemes or graphemes showed that while neither visual input induced audiovisual integration on the N1 acoustic component, both led to a supra-additive integration on P2, with a stronger integration between speech and graphemes on left-anterior electrodes. This pattern persisted on the P350 component and generalized to all electrodes. The finding suggests a strong impact of reading acquisition on phonetic processing and lexical access. It also indirectly indicates that the dynamic and predictive cues present in natural lip movements but not in static visemes are critical to the contribution of visual articulatory gestures to speech processing.
Fichier principal
Vignette du fichier
Pattamadilok&Sato_Brain&Language_MS.pdf (1.1 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03472191 , version 1 (09-12-2021)
hal-03472191 , version 2 (27-12-2021)

Identifiants

Citer

Chotiga Pattamadilok, Marc Sato. How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing. Brain and Language, 2022, 225, ⟨10.1016/j.bandl.2021.105058⟩. ⟨hal-03472191v2⟩
230 Consultations
252 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More