THE INFLUENCE OF TEXT PREPROCESSING METHODS AND TOOLS ON CALCULATING TEXT SIMILARITY

摘要

Text mining to a great extent depends on the various text preprocessing techniques. The preprocessing methods and tools which are used to prepare texts for further mining can be divided into those which are and those which are not language-dependent. The subject matter of this research was the analysis of the in?uence of these methods and tools on further text mining. We ?rst focused on the analysis of the in?uence on the reduction of the vector space model for the multidimensional represen-tation of text documents. We then analyzed the in?uence on calculating text similarity, which is the focus of this research. The conclusion we reached is that the implemen-tation of various text preprocessing methods in the Serbian language, which are used for the reduction of the vector space model for the multidimensional representation of text document, achieves the required results. But, the implementation of various text preprocessing methods speci?c to the Serbian language for the purpose of calculating text similarity can lead to great di?erences in the results.