Advancing the state of the art for handwritten math recognition: the CROHME competitions, 2011–2014

Abstract : The CROHME competitions have helped organize the field of handwritten mathematical expression recognition. This paper presents the evolution of the competition over its first 4 years, and its contributions to handwritten math recognition, and more generally structural pattern recognition research. The competition protocol, evaluation metrics and datasets are presented in detail. Participating systems are analyzed and compared in terms of the central mathematical expression recognition tasks: (1) symbol segmentation, (2) classification of individual symbols, (3) symbol relationships and (4) structural analysis (parsing). The competition led to the development of label graphs, which allow recognition results with conflicting segmentations to be directly compared and quantified using Hamming distances. We introduce structure confusion histograms that provide frequencies for incorrect subgraphs corresponding to ground-truth label subgraphs of a given size and present structure confusion histograms for symbol bigrams (two symbols with a relationship) for CROHME 2014 systems. We provide a novel analysis combining results from competing systems at the level of individual strokes and stroke pairs; this virtual merging of system outputs allows us to more closely examine limitations for current state-of-the-art systems. Datasets along with evaluation and visualization tools produced for the competition are publicly available.
Complete list of metadatas
Contributor : Harold Mouchère <>
Submitted on : Monday, May 23, 2016 - 9:07:31 AM
Last modification on : Wednesday, December 19, 2018 - 3:02:05 PM




Harold Mouchère, Richard Zanibbi, Utpal Garain, Christian Viard-Gaudin. Advancing the state of the art for handwritten math recognition: the CROHME competitions, 2011–2014. International Journal on Document Analysis and Recognition, Springer Verlag, 2016, 19 (2), pp.173--189. ⟨10.1007/s10032-016-0263-5⟩. ⟨hal-01319786⟩



Record views