Evaluation of Textual Knowledge Acquisition Tools: a Challenging Task

Haifa Zargayouna (1) , Adeline Nazarenko (1)

1 LIPN - Laboratoire d'Informatique de Paris-Nord

A large effort has been devoted to the development of textual knowledge acquisition (KA) tools, but it is still difficult to assess the progress that has been made. The lack of well-accepted evaluation protocols and data hinder the comparison of existing tools and the analysis of their advantages and drawbacks. From our own experiments in evaluating terminology and ontology acquisition tools, it appeared that the difficulties and solutions are similar for both tasks. We propose a general approach for the evaluation of textual KA tools that can be instantiated in different ways for various tasks. In this paper, we highlight the major difficulties of KA evaluation, we then present our proposal for the evaluation of terminologies and ontologies acquisition tools and the associated experiments. The proposed protocols take into consideration the specificity of this type of evaluation.

Domaines: Intelligence artificielle [cs.AI]; Traitement du texte et du document

Haifa Zargayouna, Adeline Nazarenko. Evaluation of Textual Knowledge Acquisition Tools: a Challenging Task. Proceedings of the International Conference on Language Seventh conference on International Language Resources and Evaluation (LREC 2010), May 2010, Valletta, Malta. pp.435-440. ⟨hal-00517008⟩