SpottingNet: Learning the Similarity of Word Images with Convolutional Neural Network for Word Spotting in Handwritten Historical Documents

Abstract : Word spotting is a content-based retrieval process that obtains a ranked list of word image candidates similar to the query word in digital document images. In this paper, we propose a similarity score fusion method integrated with hybrid deep-learning classification and regression models to enhance performance for Query-by-Example (QBE) word spotting. Based on the convolutional neural network end-to-end framework, the presented models enable conjointly learning of the representative word image descriptors and evaluation of the similarity measure between word descriptors directly from the word image, which are the two crucial factors in this task. In addition, we present a sample generation method using location jitter to balance similar and dissimilar image pairs and enlarge the dataset. Experiments are conducted on the classical George Washington (GW) dataset without involving any recognition methods or prior word category information. Our experiments show that the proposed model yields state-of-the-art mean average precision (mAP) of 80.03%, significantly outperforming previous results.
Complete list of metadatas

Cited literature [22 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01374401
Contributor : Harold Mouchère <>
Submitted on : Friday, May 25, 2018 - 11:22:52 AM
Last modification on : Tuesday, October 15, 2019 - 11:00:04 AM
Long-term archiving on : Sunday, August 26, 2018 - 12:45:26 PM

File

ICFHR2016-SpottingNet Learning...
Files produced by the author(s)

Identifiers

Collections

Citation

Zhuoyao Zhong, Pan Weishen, Harold Mouchère, Christian Viard-Gaudin, Jin Lianwen. SpottingNet: Learning the Similarity of Word Images with Convolutional Neural Network for Word Spotting in Handwritten Historical Documents. International Conference on Frontiers in Handwriting Recognition (ICFHR), Oct 2016, Shenzhen, China. ⟨10.1109/ICFHR.2016.58⟩. ⟨hal-01374401⟩

Share

Metrics

Record views

397

Files downloads

299