We describe TerrorCat, a submission to this year's metrics shared task. It is a machine learning-based metric that is trained on manual ranking data from WMT shared tasks 2008-2012. Input features are generated by applying automatic translation error analysis to the translation hypotheses and calculating the error category frequency differences. We additionally experiment with adding quality estimation features in addition to the error analysis-based ones. When evaluated against WMT'2012 rankings, the system-level agreement is rather high for several language pairs.
展开▼
