Skip to Main content Skip to Navigation
Journal articles

Confidence-Weighted Local Expression Predictions for Occlusion Handling in Expression Recognition and Action Unit detection

Abstract : Fully-Automatic Facial Expression Recognition (FER) from still images is a challenging task as it involves handling large interpersonal morphological differences, and as partial occlusions can occasionally happen. Furthermore, labelling expressions is a time-consuming process that is prone to subjectivity, thus the variability may not be fully covered by the training data. In this work, we propose to train Random Forests upon spatially defined local subspaces of the face. The output local predictions form a categorical expression-driven high-level representation that we call Local Expression Predictions (LEPs). LEPs can be combined to describe categorical facial expressions as well as Action Units (AUs). Furthermore, LEPs can be weighted by confidence scores provided by an autoencoder network. Such network is trained to locally capture the manifold of the non-occluded training data in a hierarchical way. Extensive experiments show that the proposed LEP representation yields high descriptive power for categorical expressions and AU occurrence prediction, and leads to interesting perspectives towards the design of occlusion-robust and confidence-aware FER systems.
Document type :
Journal articles
Complete list of metadatas

https://hal.sorbonne-universite.fr/hal-01509850
Contributor : Gestionnaire Hal-Upmc <>
Submitted on : Tuesday, April 18, 2017 - 3:48:20 PM
Last modification on : Friday, May 24, 2019 - 5:31:05 PM

Links full text

Identifiers

Citation

Arnaud Dapogny, Kévin Bailly, Séverine Dubuisson. Confidence-Weighted Local Expression Predictions for Occlusion Handling in Expression Recognition and Action Unit detection. International Journal of Computer Vision, Springer Verlag, 2016, pp.1-17. ⟨10.1007/s11263-017-1010-1⟩. ⟨hal-01509850⟩

Share

Metrics

Record views

84