End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Network

Denis Coquenet (1, 2, 3) , Clément Chatelain (3, 4) , Thierry Paquet (3, 2)

1 NU - Normandie Université
2 UNIROUEN - Université de Rouen Normandie
3 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
4 INSA Rouen Normandie - Institut national des sciences appliquées Rouen Normandie

Unconstrained handwritten text recognition remains challenging for computer vision systems. Paragraph text recognition is traditionally achieved by two models: the first one for line segmentation and the second one for text line recognition. We propose a unified end-to-end model using hybrid attention to tackle this task. We achieve state-of-the-art character error rate at line and paragraph levels on three popular datasets: 1.90% for RIMES, 4.32% for IAM and 3.63% for READ 2016. The proposed model can be trained from scratch, without using any segmentation label contrary to the standard approach. Our code and trained model weights are available at https://github.com/FactoDeepLearning/VerticalAttentionOCR.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Denis Coquenet, Clément Chatelain, Thierry Paquet. End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, ⟨10.1109/TPAMI.2022.3144899⟩. ⟨hal-03413862⟩