VIDEO SCENE SEGMENTATION OF TV SERIES USING MULTI-MODAL NEURAL FEATURES

Aman Berhe (1) , Camille Guinaudeau (2) , Claude Barras (2)

1 LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur
2 LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur

Scene segmentation of a video, a book or TV series allows them to be organized into logical story units (LSU) and is an essential step for representing, extracting and understanding their narrative structures. We propose an automatic scene segmentation method for TV series based on the grouping of adjacent shots and relying on a combination of multimodal neural features: visual features and textual features, further augmented with the temporal information which may improve the clustering of adjacent shots. Reported experiments compare the combination of different features, video frames sub-sampling and various shot clustering algorithms. The proposed method achieved good results, using different metrics, when tested on several seasons of two popular TV series.

Mots clés: Scene segmentation; TV series; Neural features; Multimodal fusion; Unsupervised

Domaines: Informatique [cs]

Aman Berhe, Camille Guinaudeau, Claude Barras. VIDEO SCENE SEGMENTATION OF TV SERIES USING MULTI-MODAL NEURAL FEATURES. Series. International Journal of TV Serial Narratives, 2019, 5 (1), ⟨10.6092/issn.2421-454X/8967⟩. ⟨hal-03298901⟩