Using BLSTM for Interpretation of 2D Languages - Case of Handwritten Mathematical Expressions

Abstract : In this work, we study how to extend the capability of BLSTM networks to process data which are not only text strings but graphical two-dimensional languages such as handwrit- ten mathematical expressions. The proposed solution aims at transforming the mathematical expression description into a sequence including at the same time symbol labels and relation- ship labels, so that classical supervised sequence labeling with recurrent neural networks can be applied. For simple one-dimensional (1-D) expression, we use the Right label to segment one symbol from the next one, as with the standard blank label for regular text. For genuine two- dimensional (2-D) expressions, we introduce additional specific labels assigned to each of the different possible spatial relationships that exist between sub-expressions. As a result, BLSTM network is able to perform at the same time the symbol recognition task and the segmentation task, which is a new perspective for the mathematical expression domain.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01576308
Contributor : Harold Mouchère <>
Submitted on : Tuesday, August 22, 2017 - 6:48:35 PM
Last modification on : Wednesday, December 19, 2018 - 3:02:05 PM

Identifiers

  • HAL Id : hal-01576308, version 1

Collections

Citation

Ting Zhang, Harold Mouchère, Christian Viard-Gaudin. Using BLSTM for Interpretation of 2D Languages - Case of Handwritten Mathematical Expressions. Colloque International Francophone sur l’Écrit et le Document 2016 (CIFED), Mar 2016, Toulouse, France. ⟨hal-01576308⟩

Share

Metrics

Record views

768