A Fast Multi-level Plagiarism Detection Method Based on Document Embedding Representation

摘要

Nowadays, global networks facilitate access to vast amount of textual information and enhance the feasibility of plagiarism as a consequence. Given the amount of text material produced everyday, the need for an automated fast plagiarism detection system is more crucial than ever. Plagiarism detection is defined as identification of reused text materials. In this regard, different algorithms have been proposed to perform the task of plagiarism detection in text documents. Due to limitation in semantic representation and computational inefficiency of traditional algorithms for plagiarism detection, in this paper, we proposed an embedding based document representation to detect plagiarism in documents using a two-level decision making approach. The method is language-independent and works properly on various languages as well. In the proposed method, words are represented as multi-dimensional vectors, and simple aggregation methods are used to combine the word vectors in order to represent sentences. By comparing representations of source and suspicious sentences, sentence pairs with the highest similarity score are considered as the candidates of the plagiarism cases. The final decision whether or not the pairs are plagiarized is taken using another level of similarity calculation using Jaccard metric by comparing the word sets of two sentences. Our method has been used in PAN2016 Persian plagiarism detection contest and results in 85.8% recall, 95.9% precision and 90.6% plagdet which is a combination of the these two measures with the measure of how concretely we retrieve plagiarism cases, on the provided data sets in a short amount of time. This method achieved the second place regarding plagdet and the first rank based on runtime.