Skip to Main content Skip to Navigation
Conference papers

A Methodology for the Comparison of Human Judgments With Metrics for Coreference Resolution

Abstract : We propose a method for investigating the interpretability of metrics used for the coreference resolution task through comparisons with human judgments. We provide a corpus with annotations of different error types and human evaluations of their gravity. Our preliminary analysis shows that metrics considerably overlook several error types and overlook errors in general in comparison to humans. This study is conducted on French texts, but the methodology should be language-independent.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03650294
Contributor : Mariya Borovikova Connect in order to contact the contributor
Submitted on : Sunday, April 24, 2022 - 8:07:26 PM
Last modification on : Tuesday, May 10, 2022 - 4:00:16 PM

File

A_Methodology_for_the_Comparis...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03650294, version 1

Citation

Mariya Borovikova, Loïc Grobol, Anaïs Lefeuvre-Halftermeyer, Sylvie Billot. A Methodology for the Comparison of Human Judgments With Metrics for Coreference Resolution. HumEval at ACL, May 2022, Dublin, Ireland. ⟨hal-03650294⟩

Share

Metrics

Record views

21

Files downloads

11