基于Hadoop的贝叶斯过滤MapReduce模型

摘要

传统分布式大型邮件系统对海量邮件的过滤存在编程难、效率低、前期训练耗用资源大等缺点，为此，对传统贝叶斯过滤算法进行并行化改进，利用云计算MapReduce模型在海量数据处理方面的优势，设计一种基于Hadoop开源云架构的贝叶斯邮件过滤MapReduce模型，优化邮件的训练和过滤过程。实验结果表明，与传统分布式计算模型相比，该模型在召回率、查准率和精确率方面性能较好，同时可降低邮件过滤成本，提高系统执行效率。%There are some disadvantages of mass mail filtering for large mail systems on the traditional distributed system including programming difficulties, low efficiency, mass system and network resources consumed. Taking advantage of the high performance of the cloud computing in processing data processing effectively, a MapReduce model of Bayesian mail filtering based on Hadoop is proposed. It improves the traditional Bayesian filtering algorithms and optimizes the mail training and filtering processes. Experimental results show that, compared with traditional distributed computing model, the Hadoop-based MapReduce model of Bayesian anti-spam mail filtering performs better in recall, precision and accuracy, reduces the cost of mail learning and classifying and improves the system efficiency.