IRISA at TRECVid 2017: Beyond Crossmodal and Multimodal Models for Video Hyperlinking

Mikail Demirdelen 1 Mateusz Budnik 1 Gabriel Sargent 1 Rémi Bois 1 Guillaume Gravier 1
1 LinkMedia - Creating and exploiting explicit links between multimedia fragments
Inria Rennes – Bretagne Atlantique , IRISA_D6 - MEDIA ET INTERACTIONS
Abstract : This paper presents the runs that were submitted to the TRECVid Challenge 2017 for the Video Hyperlinking task. The goal of the task is to propose a list of video segments, called targets, to complement query video segments defined as anchors. The data provided with the task encourage participants to make use of multiple modalities such as the audio track and the keyframes. In this context, we submitted four runs: 1) BiDNNFull uses a BiDNN model to combine ResNet with Word2Vec; 2) BiDNNFilter makes use of the same model and also exploits the metadata to narrow down the list of possible candidates; 3) BiDNNPinv tries to improve on the anchor keyframe fusion by using the Moore-Penrose pseudo-inverse and finally 4) noBiDNNPinv tests on the relevance of not using a BiDNN to fuse the modalities. Our runs were built based on a pre-trained model of ResNet as well as the transcripts and the metadata provided by the organizers of the task. The results show a gain in performance over the baseline BiDNN model both when the metadata filter was used and when the keyframe fusion was done with a pseudo-inverse.
Type de document :
Communication dans un congrès
Working Notes of the TRECVid 2017 Workshop, 2017, Gettysburg, United States
Liste complète des métadonnées

Littérature citée [14 références]  Voir  Masquer  Télécharger
https://hal.archives-ouvertes.fr/hal-01643232
Contributeur : Guillaume Gravier <>
Soumis le : mardi 21 novembre 2017 - 11:26:36
Dernière modification le : jeudi 12 avril 2018 - 01:54:52

Fichier

irisa-trecvid-lnk-2017.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01643232, version 1

Collections

INRIA | IRISA | INRIA2017 | UR1-UFR-ISTIC | UNIV-RENNES | INRIA-RENGRE | CENTRALESUPELEC

Citation

Mikail Demirdelen, Mateusz Budnik, Gabriel Sargent, Rémi Bois, Guillaume Gravier. IRISA at TRECVid 2017: Beyond Crossmodal and Multimodal Models for Video Hyperlinking. Working Notes of the TRECVid 2017 Workshop, 2017, Gettysburg, United States. 〈hal-01643232〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

647

Téléchargements de fichiers

41

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD