MoDuL: Deep Modal and Dual Landmark-wise Gated Network for Facial Expression Recognition

Sacha Bernheim (1) , Estephe Arnaud (1) , Arnaud Dapogny (2) , Kevin Bailly (1)

1 ISIR - Institut des Systèmes Intelligents et de Robotique
2 Datakalab

Automatic facial expression recognition (FER) is a challenging computer vision problem that finds a number of applications in human-computer interaction. Most recent FER approaches are deep-learning based and involve the extraction of two types of features from face images: geometric features (e.g. distances between aligned facial landmarks) and appearance features extracted using convolutional neural networks applied on patches extracted around each landmark. In this paper, we explore the use of gating networks to learn an optimal combination of these two modalities (modal gate). Furthermore, we also design landmark-wise gates to adaptively weight each landmark as well as the corresponding patch contribution. The proposed MoDuL architecture achieves state-of-the-art results on several FER databases with negligible computational overhead

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Sacha Bernheim, Estephe Arnaud, Arnaud Dapogny, Kevin Bailly. MoDuL: Deep Modal and Dual Landmark-wise Gated Network for Facial Expression Recognition. 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), Nov 2020, Buenos Aires, Argentina. pp.153-159, ⟨10.1109/FG47880.2020.00081⟩. ⟨hal-03181868⟩