An Evaluation of Inter-Annotator Agreement in the Observation of Anaphoric and Referential Relations

François Trouilleux (1, 2) , Gabriel G. Bès (1) , Éric Gaussier (2)

1 LACC - UBP - UFR Langues appliquées, Commerce, Communication - Clermont-Ferrand
2 Xerox Research Centre Europe [Meylan]

When proposing a description of the data he observes, the linguist must make sure that his observations may be also regularly made by other persons. In this paper, we introduce a typology of anaphoric and referential relations and an experiment which aims at assessing that this typology is operational. Given three newspaper articles, five students were asked to identify anaphoric and/or referential relations between expressions and referents. This inter-subjectivity test confirms results already obtained: coreference is an operational notion, but the perspicuity of other relations is not obvious.

Domaines: Informatique et langage [cs.CL]; Linguistique

François Trouilleux, Gabriel G. Bès, Éric Gaussier. An Evaluation of Inter-Annotator Agreement in the Observation of Anaphoric and Referential Relations. Third International Conference on Discourse Anaphora and Anaphor Resolution, 2000, United Kingdom. pp.1. ⟨hal-00373325⟩