Comparing formal translation evaluation and meaning-oriented translation evaluation: or how QA tools can(not) help
Abstract
This chapter puts the hypothesis that formal translation quality is representative of overall translation quality to the test. A comparison is made between the results of two evaluation methods applied to a corpus of translations carried out by Master's students. The authors compare the scores that result from a human correction (using a metric developed by the teacher-translators) with the results of an automated formal quality check with the quality assurance tool QA Distiller (QAD), developed by Yamagata. The experiment has revealed that in half of the cases, there is a rather strict correspondence between formal quality and overall quality. The comparison of the two kinds of evaluation techniques has brought to the fore the range of factors that have an impact on the differentiated scores. Even though the central hypothesis needs to be tested on more data, the experiment has shown that formal quality tools are effectively very useful to identify aspects of lack of formal quality that may be hard to spot for the human eye.