PEDIVHANDI: Multimodal Indexation and Retrieval System for Lecture Videos

Nhu van Nguyen , Franck Charneau , Nhu-Van Nguyen (1) , Jean-Marc Ogier (1)

1 L3I - Laboratoire Informatique, Image et Interaction - EA 2118

Since text in slides and teacher’s speech complementarily represent lecture contents, lecture videos can be indexed and retrieved by using a fully automatic and complete system based on the multimodal analysis of speech and text. In this paper, we present the multimodal lecture content indexing approach used in the PEDIVHANDI project. We use the discretization of speech and changes of slide’s texts to identify lecture slides in the video. We also propose a duplicate verification to remove nearly-duplicate slides. After using the Stroke Width Transfrom (SWT) text detector to obtain text regions, a standard OCR engine is used for text recognition. Finally, a context-based spell check is proposed to correct words recognized. Our system achieves the recognition precision 71% and 57% recall on a corpus of 6 presentation videos for a total duration of 8 hours.

Mots clés: Text Detection; Text Recognition; Spell Correction; Stroke Width; Lecture Video

Domaines: Informatique [cs]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Traitement du texte et du document

Nhu van Nguyen, Franck Charneau, Nhu-Van Nguyen, Jean-Marc Ogier. PEDIVHANDI: Multimodal Indexation and Retrieval System for Lecture Videos. Journal of Ambient Intelligence and Humanized Computing, 2013, 3 (4), pp.382-393. ⟨10.1007/978-3-642-37444-9_30⟩. ⟨hal-02519310⟩