HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

A study on multimodal video hyperlinking with visual aggregation

Mateusz Budnik 1 Mikail Demirdelen 1 Guillaume Gravier 1
1 LinkMedia - Creating and exploiting explicit links between multimedia fragments
Inria Rennes – Bretagne Atlantique , IRISA-D6 - MEDIA ET INTERACTIONS
Abstract : Video hyperlinking offers a way to explore a video collection, making use of links that connect segments having related content. Hyperlinking systems thus seek to automatically create links by connecting given anchor segments to relevant targets within the collection. In this paper, we further investigate multimodal representations of video segments in a hyper-linking system based on bidirectional deep neural networks, which achieved state-of-the-art results in the TRECVid 2016 evaluation. A systematic study of different input representations is done with a focus on the aggregation of the representation of multiple keyframes. This includes, in particular, the use of memory vectors as a novel aggregation technique, which provides a significant improvement over other aggre-gation methods on the final hyperlinking task. Additionally, the use of metadata is investigated leading to increased performance and lower computational requirements for the system.
Complete list of metadata

Cited literature [19 references]  Display  Hide  Download

Contributor : Guillaume Gravier Connect in order to contact the contributor
Submitted on : Monday, August 27, 2018 - 10:39:50 AM
Last modification on : Friday, April 8, 2022 - 4:08:03 PM
Long-term archiving on: : Wednesday, November 28, 2018 - 1:04:27 PM


Files produced by the author(s)



Mateusz Budnik, Mikail Demirdelen, Guillaume Gravier. A study on multimodal video hyperlinking with visual aggregation. IEEE International Conference on Multimedia and Expo, Jul 2018, San Diego, United States. pp.1-6, ⟨10.1109/ICME.2018.8486549⟩. ⟨hal-01862199⟩



Record views


Files downloads