A study on multimodal video hyperlinking with visual aggregation

Mateusz Budnik 1 Mikail Demirdelen 1 Guillaume Gravier 1
1 LinkMedia - Creating and exploiting explicit links between multimedia fragments
Inria Rennes – Bretagne Atlantique , IRISA_D6 - MEDIA ET INTERACTIONS
Abstract : Video hyperlinking offers a way to explore a video collection, making use of links that connect segments having related content. Hyperlinking systems thus seek to automatically create links by connecting given anchor segments to relevant targets within the collection. In this paper, we further investigate multimodal representations of video segments in a hyper-linking system based on bidirectional deep neural networks, which achieved state-of-the-art results in the TRECVid 2016 evaluation. A systematic study of different input representations is done with a focus on the aggregation of the representation of multiple keyframes. This includes, in particular, the use of memory vectors as a novel aggregation technique, which provides a significant improvement over other aggre-gation methods on the final hyperlinking task. Additionally, the use of metadata is investigated leading to increased performance and lower computational requirements for the system.
Liste complète des métadonnées

Littérature citée [19 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01862199
Contributeur : Guillaume Gravier <>
Soumis le : lundi 27 août 2018 - 10:39:50
Dernière modification le : jeudi 15 novembre 2018 - 11:59:01

Fichier

study-multimodal-video.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Mateusz Budnik, Mikail Demirdelen, Guillaume Gravier. A study on multimodal video hyperlinking with visual aggregation. ICME 2018 - IEEE International Conference on Multimedia and Expo, Jul 2018, San Diego, United States. IEEE, pp.1-6, 〈10.1109/ICME.2018.8486549〉. 〈hal-01862199〉

Partager

Métriques

Consultations de la notice

104

Téléchargements de fichiers

31