基于风险决策的文本语义分类算法

摘要

传统的文本分类多以空间向量模型为基础, 采用层次分类树模型进行统计分析, 该模型多数没有结合特征项语义信息, 因此可能产生大量频繁语义模式, 增加了分类路径.结合基本显露模式(eEP)在分类上的良好区分特性和基于最小期望风险代价的决策粗糙集模型, 提出了一种阈值优化的文本语义分类算法TSCTO:在获取文档特征项频率分布表之后, 首先利用粗糙集联合决策分布密度矩阵, 计算最小阈值, 提取满足一定阈值的高频词;然后结合语义分析与逆向文档频率方法获取基于语义类内文档频率的高频词;采用eEP分类方法获得最简模式;最后利用相似性公式和《知网》提供的语义相关度, 计算文本相似性得分, 利用三支决策理论对阈值进行选择.实验结果表明, TSCTO算法在文本分类的性能上有一定提升.%Most of traditional text classification algorithms are based on vector space model and hierarchical classification tree model is used for statistical analysis.The model mostly doesn't combine with the semantic information of characteristic items.Therefore it may produce a large number of frequent semantic modes and increase the paths of classification.Combining with the good distinguishment characteristic of essential Emerging Pattern (eEP) in the classification and the model of rough set based on minimum expected risk decision,a Text Semantic Classification algorithm with Threshold Optimization (TSCTO) was presented.Firstly,after obtaining the document feature frequency distribution table,the minimum threshold value was calculated by the rough set combined with distribution density matrix.Then the high frequency words of the semantic intra-class document frequency are obtained by combining semantic analysis and inverse document frequency method.In order to get the simplest model,the eEP pattern was used for classification.Finally,using similarity formula and HowNet semantic relevance degree,the score of text similarity was calculated,and some thresholds were optimized by the three-way decision theory.The experimental results show that the TSCTO algorithm has a certain improvement in the performance of text classification.