Classification of semantic relations by humans and machines

摘要

This paper addresses the classification ofsemantic relations between pairs of sentencesextracted from a Dutch parallel corpusat the word, phrase and sentence level.We first investigate the performance of humanannotators on the task of manuallyaligning dependency analyses of the respectivesentences and of assigning oneof five semantic relations to the alignedphrases (equals, generalizes, specifies, restatesand intersects). Results indicate thathumans can perform this task well, withan F-score of .98 on alignment and an Fscoreof .95 on semantic relations (aftercorrection). We then describe and evaluatea combined alignment and classificationalgorithm, which achieves an F-scoreon alignment of .85 (using EuroWordNet)and an F-score of .80 on semantic relationclassification.