A Methodological Framework for Writing Assistance Systems: Applications to Sibylle and VITIPI Systems
Résumé
This paper concerns the evaluation of Writing Assistance Systems (WAS). More precisely, it focuses on the evaluation of the linguistic process (words prediction) of these systems. We show that most of present evaluation frameworks are not really fitted to the real needs of disabled people. A methodological and a common evaluation framework of WAS is then proposed. This evaluation is based on an objective metric which is easy to compute and easy to interpret, as well as an "ecological" collection of test data (i.e. texts wrote by a disable person). This framework has been applied in an evaluation campaign that involved two systems: Sibylle and VITIPI. These experiments are described into details and results are given as well. Finally, we suggest future prospects for this evaluation framework