A Language-Independent Approach to Identify the Named Entities in Under-Resourced Languages and Clustering Multilingual Documents

摘要

This paper presents a language-independent Multilingual Document Clustering (MDC) approach on comparable corpora. Named entites (NEs) such as persons, locations, organizations play a major role in measuring the document similarity. We propose a method to iden-tify these NEs present in under-resourced Indian languages (Hindi and Marathi) using the NEs present in English, which is a. high resourced lan-guage. The identified NEs are then utilized for the formation of multilin-gual document clusters using the Bisecting k-means clustering algorithm. We didn't make use of any non-English linguistic tools or resources such as WordNet, Part-Of-Speech tagger, bilingual dictionaries, etc., which makes the proposed approach completely language-independent. Exper-iments are conducted on a standard dataset provided by FIRE~1 for their 2010 Ad-hoc Cross-Lingual document retrieval task on Indian languages. We have considered English, Hindi and Marathi news datasets for our ex-periments. The system is evaluated using F-score, Purity and Normalized Mutual information measures and the results obtained are encouraging.