SpottingNet: Learning the Similarity of Word Images with Convolutional Neural Network for Word Spotting in Handwritten Historical Documents

Zhuoyao Zhong 1 Pan Weishen 1 Harold Mouchère 2 Christian Viard-Gaudin 2 Jin Lianwen 3
1 School of Electronic and Information Engineering
2 IRCCyN - Institut de Recherche en Communications et en Cybernétique de Nantes
3 School of Electronics and Information Engineering
Abstract : Word spotting is a content-based retrieval process that obtains a ranked list of word image candidates similar to the query word in digital document images. In this paper, we propose a similarity score fusion method integrated with hybrid deep-learning classification and regression models to enhance performance for Query-by-Example (QBE) word spotting. Based on the convolutional neural network end-to-end framework, the presented models enable conjointly learning of the representative word image descriptors and evaluation of the similarity measure between word descriptors directly from the word image, which are the two crucial factors in this task. In addition, we present a sample generation method using location jitter to balance similar and dissimilar image pairs and enlarge the dataset. Experiments are conducted on the classical George Washington (GW) dataset without involving any recognition methods or prior word category information. Our experiments show that the proposed model yields state-of-the-art mean average precision (mAP) of 80.03%, significantly outperforming previous results.
Keywords : component similarit learning word spotting similarity learning similarity score fusion convolutional neural network
Document type :
Conference papers
Complete list of metadatas

Cited literature [22 references]  Display  Hide  Download
https://hal.archives-ouvertes.fr/hal-01374401
Contributor : Harold Mouchère <>
Submitted on : Friday, May 25, 2018 - 11:22:52 AM
Last modification on : Tuesday, October 15, 2019 - 11:00:04 AM
Long-term archiving on : Sunday, August 26, 2018 - 12:45:26 PM

File

ICFHR2016-SpottingNet Learning...
Files produced by the author(s)

Identifiers

Collections

UNIV-NANTES | EC-NANTES | IRCCYN | UNAM | IVC-IPI

Citation

Zhuoyao Zhong, Pan Weishen, Harold Mouchère, Christian Viard-Gaudin, Jin Lianwen. SpottingNet: Learning the Similarity of Word Images with Convolutional Neural Network for Word Spotting in Handwritten Historical Documents. International Conference on Frontiers in Handwriting Recognition (ICFHR), Oct 2016, Shenzhen, China. ⟨10.1109/ICFHR.2016.58⟩. ⟨hal-01374401⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

397

Files downloads

299

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD