A Quality Control Framework for Gold Standard Reference Translations: The Process and Toolkit Developed for GALE

摘要

While a multitude of machine translation (MT) evaluation metrics exist, most require one or more gold standard references. For the DARPA GALE program, source data is translated according to detailed guidelines and high quality standards, but these raw translations then undergo a rigorous and carefully constructed quality control (QC) process to create the final references. GALE evaluation translation references undergo several distinct phases of translation and quality control, utilizing a specially-designed toolkit, QCTool. This paper describes this framework and suggests that it may be successfully applied to improve and streamline other large-scale translation projects.