Improving retrieval performance of translation memories using morphosyntactic analyses and generalized suffix arrays

摘要

Since the 1990s, translation memory (TM) systems have been one of the most widely-used tools in the field of computer-aided translation. However, most of the commercially available systems still consider two segments to be compared similar if the sequence of characters is identical or differ only marginally in spelling. Instead, linguistic similarities are disregarded so that semantically identical or similar segments, which have different (morpho)syntactic structures, are retrieved with a lower similarity value as expected or not at all. The iMem (iMem is an abbreviation for Intelligent Translation Memories) research project aimed at improving the retrieval of those very segments by analyzing their (morpho)syntactic structure and identifying the longest common substrings (LCS) between two sentences by means of generalized suffix arrays. The results of the morphosyntactic analysis are stored in the so-called iMem-TM, an independent relational database which is connected to a commercial, non-linguistically enhanced TM via its API. Base words were used for building the suffixes to increase the probability of finding a larger number of LCS between both sentences. Furthermore, an existing algorithm for generalized suffix arrays was enhanced by an additional array in order to distinguish which suffixes derive from which sentence. In this way, identical repeating LCS within the same sentence are ignored, whereas identical repeating LCS between two different sentences are still considered. If more than one identical repeating LCS between the sentences exist, the best matching LCS is given by calculating positional differences for the identical repeating LCS and choosing the one with the minimal positional difference.