Improving Cluster Selection and Event Modeling in Unsupervised Mining for Automatic Audiovisual Video Structuring

Anh-Phuong Ta (1) , Mathieu Ben (2) , Guillaume Gravier (1)

1 TEXMEX - Multimedia content-based indexing
2 Powedia

Can we discover audio-visually consistent events from videos in a totally unsupervised manner? And, how to mine videos with different genres? In this paper we present our new results in automatically discovering audio-visual events. A new measure is proposed to select audio-visually consistent elements from the two dendrograms respectively representing hierarchical clustering results for the audio and visual modalities. Each selected element corresponds to a candidate event. In order to construct a model for each event, each candidate event is represented as a group of clusters, and a voting mechanism is applied to select training examples for discriminative classifiers. Finally, the trained model is tested on the entire video to select video segments that belong to the event discovered. Experimental results on different and challenging genres of videos, show the effectiveness of our approach.

Mots clés: Video mining; Video structuring; Multimodality; Mutual Information; Event discovery; Structural event; Audiovisual consistency

Domaines: Multimédia [cs.MM]

Anh-Phuong Ta, Mathieu Ben, Guillaume Gravier. Improving Cluster Selection and Event Modeling in Unsupervised Mining for Automatic Audiovisual Video Structuring. MMM - 18th International Conference on MultiMedia Modeling, 2012, Austria. ⟨hal-00671157⟩