Cultural Differences in Pattern Matching: Multisensory Recognition of Socio-affective Prosody

Takaaki Shochi (1) , Jean-Luc Rouas (2) , Marine Guerry (1) , Donna Erickson (3)

1 CLLE-ERSS - Cognition, Langues, Langage, Ergonomie
2 LaBRI - Laboratoire Bordelais de Recherche en Informatique
3 Sophia University [Tokyo]

This study focuses on the cross-cultural differences in perception of audio visual prosodic recordings of Japanese social affects. The study compares cultural differences of perceptual patterns of 21 Japanese subjects with 20 French subjects who have no knowledge of Japanese language or Japanese social affects. The test material is a semantically affectively neutral utterance expressed in 9 various social affects by 2 Japanese speakers (one male, one female) who were chosen as best performers in our previous recognition experiment. The task was to create a specific audiovisual affect by choosing one video stimuli among 9 choices and one audio stimuli, again among 9 choices. The participants could preview each audio and video stimuli individually and also the combination of chosen stimuli. The results reveal that native subjects can correctly combine auditory and visually expressed social affects, showing some confusion inside semantic categories. Different matching patterns are observed for non-native subjects especially for a type of cultural-specific politeness.

Mots clés: Multisensory recognition; Pattern matching; Cultural difference

Domaines: Traitement du signal et de l'image [eess.SP]

Takaaki Shochi, Jean-Luc Rouas, Marine Guerry, Donna Erickson. Cultural Differences in Pattern Matching: Multisensory Recognition of Socio-affective Prosody. Interspeech 2018, Sep 2018, Hyderabad, India. ⟨10.21437/interspeech.2018-1795⟩. ⟨hal-01913705⟩