Magnifying Subtle Facial Motions for Effective 4D Expression Recognition

Qingkai Zhen (1) , Di Huang (2) , Hassen Drira (2) , Boulbaba Ben Amor (2) , Yunhong Wang (1) , Mohamed Daoudi (2)

1 BUAA - Beihang University
2 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189

In this paper, an effective approach is proposed for automatic 4D Facial Expression Recognition (FER). It combines two growing but disparate ideas in the domain of computer vision, i.e. computing spatial facial deformations using a Riemannian method and magnifying them by a temporal filtering technique. Key frames highly related to facial expressions are first extracted from a long 4D video through a spectral clustering process, forming the Onset-Apex-Offset flow. It is then analyzed to capture the spatial deformations based on Dense Scalar Fields (DSF), where registration and comparison of neighboring 3D faces are jointly led. The generated temporal evolution of these deformations is further fed into a magnification method to amplify facial activities over time. The proposed approach allows revealing subtle deformations and thus improves the emotion classification performance. Experiments are conducted on the BU-4DFE and BP-4D databases, and competitive results are achieved compared to the state-of-the-art.

Mots clés: Riemannian geometry; Index Terms—4D Facial Expression Recognition; subtle motion magnification; key frame detection

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Qingkai Zhen, Di Huang, Hassen Drira, Boulbaba Ben Amor, Yunhong Wang, et al.. Magnifying Subtle Facial Motions for Effective 4D Expression Recognition. IEEE Transactions on Affective Computing, 2019, 10 (4), pp.524-536. ⟨hal-01577604⟩