Automated video analysis of emotion and dystonia in epileptic seizures

Jen-Cheng Hou (1) , Monique Thonnat (1) , Fabrice Bartolomei (2, 3) , Aileen Mcgonigal (2, 3)

1 STARS - Spatio-Temporal Activity Recognition Systems
2 APHM - Assistance Publique - Hôpitaux de Marseille
3 INS - Institut de Neurosciences des Systèmes

To investigate the accuracy of deep learning methods applied to seizure video data, in discriminating individual semiologic features of dystonia and emotion in epileptic seizures. A dataset of epileptic seizure videos was used from patients explored with stereo-EEG for focal pharmacoresistant epilepsy. All patients had hyperkinetic (HKN) seizures according to ILAE definition. Presence or absence of (1) dystonia and (2) emotional features in each seizure was documented by an experienced clinician. A deep learning multi-stream model with appearance and skeletal keypoints, face and body information, using graph convolutional neural networks, was used to test discrimination of dystonia and emotion. Classification accuracy was assessed using a leave-one-subject-out analysis. We studied 38 HKN seizure videos in 19 patients. By visual analysis based on ILAE criteria, 9/19 patients were considered to have dystonia and 9/19 patients were considered to have emotional signs. Two patients had both dystonia and emotional signs. Applying the deep learning multistream model, spatiotemporal features of facial appearance showed best accuracy for emotion detection (F1 score 0.84), while skeletal keypoint detection performed best for dystonia (F1 score 0.83). Here, we investigated deep learning of video data for analyzing individual semiologic features of dystonia and emotion in hyperkinetic seizures. Automated classification of individual semiologic features is possible and merits further study.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Jen-Cheng Hou, Monique Thonnat, Fabrice Bartolomei, Aileen Mcgonigal. Automated video analysis of emotion and dystonia in epileptic seizures. Epilepsy Research, 2022, 184, ⟨10.1016/j.eplepsyres.2022.106953⟩. ⟨hal-03817305⟩