The Evaluation and Systems Analysis of the SYSTRAN Machine Translation System.

摘要

This report is the product of contractual effort to determine how machine translations produced by SYSTRAN can be improved upon and where to integrate the required correctional procedures. Two tasks were pursued: the first a survey of SOTA in translation evaluation methods; the second, the actual determination of translation deficiencies and identification of correctional procedures. The survey did not produce any new methods of evaluation, and it was already known that the old procedures tested quality characteristics but did not identify deficiencies. Consequently, a method was adopted which did meet the requirements to achieve the objectives of the contractual effort. Russian source material from Mechanical Engineering and Metallurgy were selected for translation using a base-line SYSTRAN configuration. One sample of each was translated and then corrected by a bilingual expert in each field. Two types of corrections were considered implementable, stem dictionary update and semantic expression update. The same samples were re-translated incorporating first the updated stem dictionary and then both it and the updated semantic expressions dictionary. Improvements to sentences under each condition were recorded. also, a different sample from each field was translated by the base-1 line configuration and each updated version to maintain control of carry-over effects that may occur due to updating. Overall results across both subject areas show a 50% improvement in the sample of sentences when the stem dictionary is updated and 56% when the semantic expression update is included in translation.