Somatosensory information affects word segmentation and perception of lexical information

Rintaro Ogane (1) , Jean-Luc Schwartz (1) , Takayuki Ito (1, 2)

1 GIPSA-PCMD - GIPSA - Perception, Contrôle, Multimodalité et Dynamiques de la parole
2 Haskins Laboratories

In the framework of perceptuo-motor theories of speech perception, it has been argued that speech articulation could play a role in speech perception. Indeed, recent finding demonstrates that somatosensory inputs associated with speaking motion changes the perception of speech sounds. However, it is still unknown whether somatosensory effects could go up to the level of lexical access in speech comprehension. Access to lexical information depends on a complex word segmentation process. This study aims to examine whether segmentation and lexical information processing could be changed by somatosensory inputs associated with facial skin deformation. We here focus on “elisions” between definite article and noun in French (e.g. “l’affiche” [the poster] or “la fiche” [the form]), which have the same pronunciation, but can be differentiated by hyper-articulation for the production of the first vowel in each word. If somatosensory information plays a role in lexical information processing, we reasoned that the perception of such sequences could be changed if somatosensory stimulation was applied in an appropriate timing associated to the corresponding speech gesture. To test this, we applied somatosensory perturbation associated with facial skin deformation at different timings along the presentation of the auditory stimulation. We tested native speakers of French who performed a two-alternative forced choice task between the two percepts associated to a single speech sequence. The stimulation sound was presented through headphones within the carrier phrase “C’est ___ (This is ___)”. Participants identified which word was presented (i.e. “affiche” vs. “fiche”) by pressing the left or right arrow button on the keyboard as quickly as possible. We used 17 different ambiguous speech sentences recorded by a native French speaker. Utterances were pronounced neutrally without adding hyper-articulation in any single vowel, that is, removing as much as possible acoustic information for decision. For each audio sentence, we applied a somatosensory stimulation consisting in facial skin stretch perturbation, generated by a robotic device. We varied the time of presentation of this facial skin stretch perturbation, with 8 different temporal positions within an acoustic sentence. These temporal positions were set relative to the timing of the first vowel peak amplitude (i.e. “a” in the previous example), separated by 100 ms intervals from -400 ms (around the vowel in “c’est”) to 300 ms (around the second vowel, i.e. “i” in the previous example). In each combination of audio and somatosensory stimulation, four subject’s responses were recorded, with a total of 544 stimuli (17 sentences * 8 timing conditions * 4 repetitions) presented in a random order. The judgement probability (i.e. percentage of “la fiche” responses) was calculated for each subject and each timing condition. It appears that the percentage of judgement probability was reduced when somatosensory stimulation was ahead of the first vowel (more “l’affiche” responses), and increased when somatosensory stimulation was delayed between the first and second vowel (more “la fiche” responses). This suggests that somatosensory information intervenes in the processing of lexical information, which corresponds to a relatively higher level of processing of speech perception.

Domaines: Sciences cognitives

Rintaro Ogane, Jean-Luc Schwartz, Takayuki Ito. Somatosensory information affects word segmentation and perception of lexical information. SNL 2017 - 9th Annual Society for the Neurobiology of Language Conference, Nov 2017, Baltimore, Maryland, United States. ⟨hal-01658527⟩