Towards a Leaner Evaluation Process: Application to Error Correction Systems

Arnaud Renard 1 Sylvie Calabretto 1 Béatrice Rumpler 1
1 DRIM - Distribution, Recherche d'Information et Mobilité
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : While they follow similar procedures, evaluations of state of the art error correction systems always rely on different resources (collections of documents, evaluation metrics, dictionaries, ...). In this context, error correction approaches cannot be directly compared without being re-implemented from scratch every time they have to be compared with a new one. In other domains such as Information Retrieval this problem is solved through Cranfield like experiments such as TREC evaluation campaign. We propose a generic solution to overcome those evaluation difficulties through a modular evaluation platform which formalizes similarities between evaluation procedures and provides standard sets of instantiated resources for particular domains. While this was our main problem at first, in this article, the set of resources is dedicated to the evaluation of error correction systems. The idea is to provide the leanest way to evaluate error correction systems by implementing only the core algorithm and relying on the platform for everything else.
Document type :
Book sections
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01562810
Contributor : Équipe Gestionnaire Des Publications Si Liris <>
Submitted on : Monday, July 17, 2017 - 9:53:46 AM
Last modification on : Friday, January 11, 2019 - 4:35:34 PM

Identifiers

  • HAL Id : hal-01562810, version 1

Citation

Arnaud Renard, Sylvie Calabretto, Béatrice Rumpler. Towards a Leaner Evaluation Process: Application to Error Correction Systems. Lecture Notes in Business Information Processing (LNBIP), Springer-Verlag, 2013. ⟨hal-01562810⟩

Share

Metrics

Record views

77