An audio-visual approach to web video categorization

Bogdan Ionescu (1) , Klaus Seyerlehner (2) , Ionut Mironica (1) , Constantin Vertan (1) , Patrick Lambert (3)

1 LAPI - Laboratorul de Analiza si Prelucrarea Imaginilor [Bucarest]
2 Department of Computational Perception
3 LISTIC - Laboratoire d'Informatique, Systèmes, Traitement de l'Information et de la Connaissance

In this paper we address the issue of automatic video genre categorization of web media using an audio-visual approach. To this end, we propose content descriptors which exploit audio, temporal structure and color information. The potential of our descriptors is experimentally validated both from the perspective of a classification system and as an information retrieval approach. Validation is carried out on a real scenario, namely on more than 288 hours of video footage and 26 video genres specific to blip.tv media platform. Additionally, to reduce semantic gap, we propose a new relevance feedback technique which is based on hierarchical clustering. Experimental tests prove that retrieval performance can be significantly increased in this case, becoming comparable to the one obtained with high level semantic textual descriptors.

Mots clés: Audio block-based descriptors; Color perception; Action assessment; Video relevance feedback; Video genre classification

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Bogdan Ionescu, Klaus Seyerlehner, Ionut Mironica, Constantin Vertan, Patrick Lambert. An audio-visual approach to web video categorization. Multimedia Tools and Applications, 2014, 70 (2), pp. 1007-1032. ⟨10.1007/s11042-012-1097-x⟩. ⟨hal-00732716⟩