Emotional State Classification using Pulse Rate Variability

R. Meziati Sabour (1) , Yannick Benezeth (1) , F. Marzani (1) , K. Nakamura (2) , R. Gomez (2) , F. Yang (1)

1 ImViA - Imagerie et Vision Artificielle [Dijon]
2 Honda Research Institute

Humans are continually exposed to emotional stimuli. Gesture, voice intonation, and facial expressions are among the most popular cues that describe our changing emotions. However, the physiological systems that govern our bodily functions are also impacted by the different emotions that we feel. Psychophysiology is a science concerned with the existing relationship between the psychic state of a person and the physiological signals their body emits. In fact, this connection is due to the autonomic nervous system activity, whose sympathetic nerves get sparked when a person is emotionally excited. One particular physiological phenomenon has turned out to be an excellent indicator of the autonomic function. It is the spontaneous fluctuations in the heart rhythms, which can be described by the pulse rate variability (PRV). In this work, the PRV is obtained using remote photoplethysmography. We prove that from a simple RGB camera, it is possible to assess the emotional state of a person by analysing their pulse rate variations. This optimistic finding is supported by surprising results and an accuracy rate of around 60% on the CAS(ME) 2 dataset. This is the first study to propose an emotion classification based on a physiological signal analysis using CAS(ME) 2 .

Mots clés: autonomic function; pulse rate variability; PRV; emotion classification; physiological signal analysis; emotional state classification; emotional stimuli; voice intonation; facial expressions; physiological systems; psychic state; autonomic nervous system activity; sympathetic nerves; RGB camera; CAS(ME)2 dataset; remote photoplethysmography; autonomic nervous system; cardiology; physiological phenomenon; psychology; Skin; Physiology; Resonant frequency; Biomedical monitoring; emotion recognition; image colour analysis; medical signal processing; neurophysiology; photoplethysmography; Feature extraction; Heart rate variability

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Traitement du signal et de l'image [eess.SP]

R. Meziati Sabour, Yannick Benezeth, F. Marzani, K. Nakamura, R. Gomez, et al.. Emotional State Classification using Pulse Rate Variability. 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP), Jul 2019, Wuxi, China. pp.86-90, ⟨10.1109/SIPROCESS.2019.8868781⟩. ⟨hal-02438267⟩