A Methodological Framework for Writing Assistance Systems: Applications to Sibylle and VITIPI Systems

摘要

This paper concerns the evaluation of Writing Assistance Systems (WAS). More precisely, it focuses on the evaluation of the linguistic process (word prediction) of these systems. We show that most of present evaluation frameworks are not really fitted to the real needs of disabled. A methodological and a common evaluation framework of WAS. is then proposed. This evaluation is based on an objective metric which is easy to compute andto interpret, as well as an "ecological" collection of test data (i.e. texts wrote by a disableb people). This framework has been applied in an evaluation campaign that involved two systems: Sibylle and VITIPI. These experiments are described into details and results are given as well. Finally, we suggest future prospects for this evaluation framework.