Skip to Main content Skip to Navigation
Journal articles

How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing

Abstract : Articulatory gestures and orthography are connected with speech through a natural and an artificial association, respectively. This EEG study investigated whether the integrations between speech and these two visual inputs rely on the same mechanism, despite their different characteristics. A comparison of skilled readers' brain responses elicited by spoken words presented alone versus synchronously with visemes or graphemes showed that while neither visual input induced audiovisual integration on the N1 acoustic component, both led to a supra-additive integration on P2, with a stronger integration between speech and graphemes on left-anterior electrodes. This pattern persisted on the P350 component and generalized to all electrodes. The finding suggests a strong impact of reading acquisition on phonetic processing and lexical access. It also indirectly indicates that the dynamic and predictive cues present in natural lip movements but not in static visemes are critical to the contribution of visual articulatory gestures to speech processing.
Document type :
Journal articles
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03472191
Contributor : Chotiga Pattamadilok Connect in order to contact the contributor
Submitted on : Monday, December 27, 2021 - 11:32:26 AM
Last modification on : Friday, February 4, 2022 - 3:26:12 AM

Files

Pattamadilok&Sato_Brain&Langua...
Files produced by the author(s)

Identifiers

Collections

Citation

Chotiga Pattamadilok, Marc Sato. How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing. Brain and Language, Elsevier, 2022, 225, ⟨10.1016/j.bandl.2021.105058⟩. ⟨hal-03472191v2⟩

Share

Metrics

Record views

292

Files downloads

139