Evaluating machine translation for assimilation via a gap-filling task

摘要

This paper provides additional observations on the viability of a strategy independently proposed in 2012 and 2013 for evaluation of machine translation (MT) for assimilation purposes. The strategy involves human evaluators, who are asked to restore keywords (to fill gaps) in reference translations. The evaluation method is applied to two language pairs, Basque-Spanish and Tatar-Russian. To reduce the amount of time required to prepare tasks and analyse results, an open-source task management system is introduced. The evaluation results show that the gap-filling task may be suitable for measuring MT quality for assimilation purposes.