CRF based context modeling for person identification in broadcast videos

Abstract : We are investigating the problem of speaker and face identification in broadcast videos. Identification is performed by associating automatically extracted names from overlaid texts with speaker and face clusters. We aimed at exploiting the structure of news videos to solve name/cluster association ambiguities and clustering errors. The proposed approach combines iteratively two conditional random fields (CRF). The first CRF performs the person diarization (joint temporal segmentation, clustering, and association of voices and faces) jointly over the speech segments and the face tracks. It benefits from contextual information being extracted from the image backgrounds and the overlaid texts. The second CRF associates names with person clusters, thanks to co-occurrence statistics. Experiments conducted on a recent and substantial public dataset containing reports and debates demonstrate the interest and complementarity of the different modeling steps and information sources: the use of these elements enables us to obtain better performances in clustering and identification, especially in studio scenes.
Type de document :
Article dans une revue
Frontiers in information and communication technologies, Frontiers Media S.A., 2016, 3, pp.9. 〈http://journal.frontiersin.org〉. 〈10.3389/fict.2016.00009〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01433154
Contributeur : Sylvain Meignier <>
Soumis le : mardi 21 mars 2017 - 23:20:05
Dernière modification le : jeudi 6 avril 2017 - 10:15:27
Document(s) archivé(s) le : jeudi 22 juin 2017 - 14:58:49

Fichier

Gay_FRONTIERS-CIA_2016.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Paul Gay, Sylvain Meignier, Jean-Marc Odobez, Paul Deléglise. CRF based context modeling for person identification in broadcast videos. Frontiers in information and communication technologies, Frontiers Media S.A., 2016, 3, pp.9. 〈http://journal.frontiersin.org〉. 〈10.3389/fict.2016.00009〉. 〈hal-01433154〉

Partager

Métriques

Consultations de la notice

126

Téléchargements de fichiers

34