Statistical Topic Modeling for Urdu Text Articles

摘要

Natural Language Processing (NLP) is a branch of Artificial Intelligence to help computers manipulate and interpret human languages. In NLP, text mining is a technique to derive useful information from text. Topic Model (TM) is a statistical model to extract topics from a large collection of unlabeled text using NLP and machine learning techniques. Several effective TM are available to fulfill the needs of various languages like English, German, Arabic etc. However no compelling TM is available for poor resource South Asian language Urdu. In this research study, our focus is to work on existing TM like Latent Dirichlet Allocation (LDA) to overcome the issues of Urdu language in text mining. We studied and analyzed LDA as an unsupervised model for the Urdu topic identification. Hence, we studied LDA deeply for Urdu topic identification at two levels: Variational Bayes (VB) based LDA for Urdu (VB-ULDA) with stemmer and without stemmer. Experiments are performed on a self-created massive number of Urdu documents in four different corpora. Experimental study shows that VB-ULDA outperformed in the identification of topics from Urdu text documents as compared to existing Urdu LDA (ULDA) in terms of accuracy and efficiency and results also reveal the high impact of stemming algorithm in Urdu topic identification.