Text content and task performance in the evaluation of a Natural Language Generation system
Résumé
An important question in the evaluation of Natural Language Generation systems concerns the relationship between textual characteristics and task performance. If the results of task-based evaluation can be correlated to properties of the text, there are better prospects for improving the system. The
present paper investigates this relationship by focusing on the outcomes of a task-based evaluation of a system that generates summaries of patient
data, attempting to correlate these with the results of an analysis of the system’s texts, compared to a set of gold standard human-authored summaries.