Strength Modelling for Real-World Automatic Continuous Affect Recognition from Audiovisual Signals

Abstract : Automatic continuous affect recognition from audiovisual cues is arguably one of the most active research areas in machine learning. In addressing this regression problem, the advantages of the models, such as the global-optimisation capability of Support Vector Machine for Regression and the context-sensitive capability of memory-enhanced neural networks, have been frequently explored, but in an isolated way. Motivated to leverage the individual advantages of these techniques, this paper proposes and explores a novel framework, Strength Modelling, where two models are concatenated in a hierarchical framework. In doing this, the strength information of the first model, as represented by its predictions, is joined with the original features, and this expanded feature space is then utilised as the input by the successive model. A major advantage of Strength Modelling, besides its ability to hierarchically explore the strength of different machine learning algorithms, is that it can work together with the conventional feature-and decision-level fusion strategies for multimodal affect recognition. To highlight the effectiveness and robustness of the proposed approach, extensive experiments have been carried out on two time-and value-continuous spontaneous emotion databases (RECOLA and SEMAINE) using audio and video signals. The experimental results indicate that employing Strength Modelling can deliver a significant performance improvement for both arousal and valence in the unimodal and bimodal settings. The results further show that the proposed systems is competitive or outperform the other state-of-the-art approaches, but being with a simple implementation.
Type de document :
Article dans une revue
Image and Vision Computing, Elsevier, 2017, 65, pp.76-86. 〈10.1016/j.imavis.2016.11.020〉
Liste complète des métadonnées

Littérature citée [49 références]  Voir  Masquer  Télécharger
Contributeur : Fabien Ringeval <>
Soumis le : jeudi 9 mars 2017 - 18:08:30
Dernière modification le : mardi 12 février 2019 - 01:31:18
Document(s) archivé(s) le : samedi 10 juin 2017 - 15:22:22


Fichiers produits par l'(les) auteur(s)


Domaine public




Jing Han, Zixing Zhang, Nicholas Cummins, Fabien Ringeval, Björn Schuller. Strength Modelling for Real-World Automatic Continuous Affect Recognition from Audiovisual Signals. Image and Vision Computing, Elsevier, 2017, 65, pp.76-86. 〈10.1016/j.imavis.2016.11.020〉. 〈hal-01486186〉



Consultations de la notice


Téléchargements de fichiers