4D Facial Expression Recognition by Learning Geometric Deformations

Boulbaba Ben Amor (1, 2) , Hassen Drira (2, 1) , Stefano Berretti (3) , Mohamed Daoudi (2, 1) , Anuj Srivastava (4)

1 Institut TELECOM/TELECOM Lille1
2 LIFL - FOX MIIRE
3 DSI - Dipartimento di Sistemi e Informatica
4 Department of Statistics [Tallahassee, FL]

In this paper, we present an automatic approach for facial expression recognition from 3D video sequences. In the proposed solution, the 3D faces are represented by collections of radial curves and a Riemannian shape analysis is applied to effectively quantify the deformations induced by the facial expressions, in a given subsequence of 3D frames. This is obtained from the \textit{Dense Scalar Field}, which denotes the shooting directions of the geodesic paths constructed between pairs of corresponding radial curves of two faces. As the resulting Dense Scalar Fields show a high dimensionality, LDA transformation is applied to the dense feature space. Two methods are then used for classification: (i) 3D motion extraction with temporal HMM modeling; and (ii) Mean deformation capturing with Random Forest. While a dynamic HMM on the features is trained in the first approach, the second one computes mean deformations under a window and applies multi-class Random Forest. Both of the proposed classification schemes on the scalar fields showed comparable results and outperformed earlier studies on facial expression recognition from 3D video sequences.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Apprentissage [cs.LG]

Boulbaba Ben Amor, Hassen Drira, Stefano Berretti, Mohamed Daoudi, Anuj Srivastava. 4D Facial Expression Recognition by Learning Geometric Deformations. IEEE Transactions on Cybernetics, 2014, 44 (12), pp.2443-2457. ⟨hal-00949002⟩