Skip to Main content Skip to Navigation
New interface
Conference papers

SMAD: A tool for automatically annotating the smile intensity along a video record

Abstract : We present an automatic tool for tracing the dynamic of the smile intensity along a video record. The processed output consist in a sequence of adjusted time intervals labeled following the Smiling Intensity Scale of Gironzetti et al. (2016) [3], a 5 levels scale varying from neutral facial expression to laughing smile. The state-of-the-art toolbox OpenFace [2] is firstly used for tracking the face and for measuring the intensities of the facial Action Units of interest all along the video. In a second step the smile intensity automatic annotation is performed based on these OpenFace Action Units measurements. The statistical model underlying our SMAD tool is trained on a 1 hour manually annotated smiles of the CHEESE! corpus [4] (a full description of the model will be found in Rauzy & Amoyal, submitted to JMUI).
The evaluation of the engine reveals an observed agreement of 68% between manual and automatic annotations. A more concrete experiment conducted on in-the-wild video records shows that manually correcting the labels and interval boundaries of the automatic outputs reduces by a factor 10 the annotation time as compared with the time spent for manually annotating smile intensities without pretreatment. The smile annotation of PACO [1], a 5 hours corpus of conversational data built up for analyzing the impact of common ground in spontaneous face-to-face interaction, has already benefited from this gain in annotation time.
The SMAD scripts and documentation are available to download at the HMAD open source project url page https : //github:com/srauzy/HMAD.

[1] Amoyal M, Priego-Valverde B, Rauzy S (2020) PACO : A corpus to analyze the impact of common ground in spontaneous face-to-face interaction. In: Language Resources and Evaluation Conference, LREC 2020, Marseille, France
[2] Baltruisaitis T, Zadeh A, Lim YC, Morency LP (2018) Openface 2.0: Facial behavior analysis toolkit. In: 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), pp 59-66
[3] Gironzetti E, Attardo S, Pickering L (2016) Smiling, gaze, and humor in conversation: A pilot study. In: Ruiz-Gurillo L (ed) Metapragmatics of Humor: Current research trends, pp 235-254
[4] Priego-Valverde B, Bigi B, Attardo S, Pickering L, Gironzetti E (2018) Is smiling during humor so obvious? A cross-cultural comparison of smiling behavior in humorous sequences in American English and French interactions. Intercultural Pragmatics
Complete list of metadata
Contributor : Stéphane RAUZY Connect in order to contact the contributor
Submitted on : Thursday, April 2, 2020 - 12:00:13 PM
Last modification on : Friday, June 3, 2022 - 3:10:51 AM


  • HAL Id : hal-02529371, version 1



Stéphane Rauzy, Mary Amoyal. SMAD: A tool for automatically annotating the smile intensity along a video record. HRC2020, 10th Humour Research Conference, Mar 2020, Commerce, Texas, United States. ⟨hal-02529371⟩



Record views


Files downloads