Vers l’Alignement des Signaux Écrit et Sonore : Application pour la reconnaissance des expressions mathématiques

Abstract : In this paper , we report some new results related to mathematical expression recognition. We tackle this problem, known to be very difficult, using multimodal information – handwriting and speech –. This bimodality aspect of the information provides greater reliability since the modalities in concern are very complementary. To combine the signals coming from both modalities in an efficient way , we propose an original learning process based on neural networks. This approach allows not only fusing both streams, but also the alignment of the signals coming from both modalities. The bimodal system is evaluated on real bimodal data fr om the HAMEX dataset and the obtained results are compared to a single modality (handwriting) based system.
Document type :
Conference papers
Complete list of metadatas

Cited literature [9 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01151106
Contributor : Harold Mouchère <>
Submitted on : Tuesday, May 12, 2015 - 2:24:05 PM
Last modification on : Wednesday, December 19, 2018 - 3:02:05 PM
Long-term archiving on : Monday, September 14, 2015 - 11:11:36 PM

File

CIFED-25.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01151106, version 1

Citation

Sofiane Medjkoune, Harold Mouchère, Simon Petitrenaud, Christian Viard-Gaudin. Vers l’Alignement des Signaux Écrit et Sonore : Application pour la reconnaissance des expressions mathématiques. Colloque International Francophone sur l’Écrit et le Document 2014 (CIFED), Mar 2014, Nancy, France. pp.341-356. ⟨hal-01151106⟩

Share

Metrics

Record views

302

Files downloads

452