A Link Strength Based Language Model for Chinese Short-text Categorization

摘要

For the sake of further mining information of high discrimination in Chinese short-text, this paper presents a link strength based language model, which could epurate the intension of the short-text fully by considering term relations and order deeply and remedy the deficiency that the short-text can't express information adequately. The proposed model changes the link structure in traditional language mode that could reduce information loss brought about by conventional categorization model and suits for short text classification. This is implemented by introducing a new scale-cumulative link distribution of depicting the link strength between terms, and it solves the problem that traditional language mode assumes all adjacent terms are related, but in practice it doesn't accord to the real fact. Compared with traditional language model, experimental results show the superiority of the proposed model in Chinese short-text categorization.