Skip to Main content Skip to Navigation
Journal articles

Affect in Multimedia: Benchmarking Violent Scenes Detection

Abstract : In this paper, we report on the creation of a publicly available, common evaluation framework for Violent Scenes Detection (VSD) in Hollywood and YouTube videos. We propose a robust data set, the VSD96, with more than 96 hours of video of various genres, annotations at different levels of detail (e.g., shot-level, segment-level), annotations of mid-level concepts (e.g., blood, fire), various pre-computed multi-modal descriptors, and over 230 system output results as baselines. This is the most comprehensive data set available to this date tailored to the VSD task and was extensively validated during the MediaEval benchmarking campaigns. Furthermore, we provide an in-depth analysis of the crucial components of VSD algorithms, by reviewing the capabilities and the evolution of existing systems (e.g., overall trends and outliers, the influence of the employed features and fusion techniques, the influence of deep learning approaches). Finally, we discuss the possibility of going beyond state-of-the-art performance via an ad-hoc late fusion approach. Experimentation is carried out on the VSD96 data. We provide the most important lessons learned and gained insights. The increasing number of publications using the VSD96 data underline the importance of the topic. The presented and published resources are a practitioner's guide and also a strong baseline to overcome, which will help researchers for the coming years in analyzing aspects of audiovisual affect and violence detection in movies and videos.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03032718
Contributor : Guillaume Gravier Connect in order to contact the contributor
Submitted on : Tuesday, December 1, 2020 - 9:57:23 AM
Last modification on : Wednesday, November 3, 2021 - 8:18:57 AM
Long-term archiving on: : Tuesday, March 2, 2021 - 6:33:00 PM

File

VSD_TAC.pdf
Files produced by the author(s)

Identifiers

Citation

Mihai Gabriel Constantin, Liviu Daniel Stefan, Bogdan Ionescu, Claire-Helene Demarty, Mats Sjoberg, et al.. Affect in Multimedia: Benchmarking Violent Scenes Detection. IEEE Transactions on Affective Computing, Institute of Electrical and Electronics Engineers, 2020, ⟨10.1109/TAFFC.2020.2986969⟩. ⟨hal-03032718⟩

Share

Metrics

Record views

54

Files downloads

272