Recognition of Online Handwritten Mathematical Expressions using Contextual Information

Abstract : Online handwritten mathematical expressions consist of sequences of strokes. Automatic recognition these data requires solving three subproblems: symbol segmentation, symbol classification, and structural analysis (i.e. identification of spatial relations between symbols). Ambiguity, that often leads to several likely interpretations, and the non-linear structure of the expressions are main issues of the recognition process. In this thesis, we model the recognition problem as a graph parsing problem. The graph-based description of relations in production rules allows direct modeling of non-linear structures. Our parsing algorithm determines recursive partitions of the input strokes that induce graphs matching the production rule graphs. To mitigate the computational cost, we constrain the partitions to graphs derived from sets of symbol and relation hypotheses, calculated using previously trained classifiers. A set of labels that indicate likely interpretations is associated to each hypothesis, and the selection of the best interpretation is driven by the parsing algorithm. The parsing method computes multiple parse trees to represent alternative interpretations, assigns a cost to each tree and selects a tree with minimum cost as result. The evaluations show that the proposed method is more accurate than several state of the art methods; the use of symbol and relation hypotheses to constrain the search space effectively reduces the parsing complexity; and adaptation to other two-dimensional object recognition problems is possible. As a secondary contribution, we developed a framework to automatize the handwritten mathematical expression datasets building process.
Complete list of metadatas

Cited literature [80 references]  Display  Hide  Download
Contributor : Harold Mouchère <>
Submitted on : Friday, June 3, 2016 - 2:53:42 PM
Last modification on : Monday, February 24, 2020 - 2:13:47 PM


  • HAL Id : tel-01326354, version 1



Frank Julca-Aguilar. Recognition of Online Handwritten Mathematical Expressions using Contextual Information. Computer Vision and Pattern Recognition [cs.CV]. Université de Nantes; Université Bretagne Loire; Universidade de São Paulo, 2016. English. ⟨tel-01326354⟩



Record views


Files downloads