基于Hadoop的社交网络服务推荐算法

摘要

In order to process huge amount of data generated in the social network with efficiency and scalability,we designed the distributed TF-IDF(Term Frequency-Inverse Document Frequency)algorithm suitable for MapReduce,and implemented this algorithm on Hadoop.This algorithm extracts key words in user's weibo,in this way user's interest could be found,and the corresponding service could be recommended to the user.In order to verify the validity and scalability of the distributed TF-IDF algorithm,the results of the distributed TF-IDF algorithm and TextRank algorithm was compared.The experimental results show that key words extracted bythe distributed TF-IDF algorithm could represent characteristics of the user more accurately.By Contrasting the response time,it could be seen that the distributed TF-IDF algorithm has a good scalability.%为高效处理社交网络产生的海量数据,并保证社交网的可扩展性,将TF-IDF(Term Frequency-Inverse Document Frequency)算法进行MapReduce化设计,并在Hadoop云平台上实现分布式的TF-IDF算法.利用该算法提取用户微博中的关键词,再根据关键词发现用户的兴趣,并对用户做相应的推荐.为验证分布式TF-IDF算法的有效性和可扩展性,与TextRank算法的结果做对比.实验结果表明,分布式TF-IDF算法提取的关键词能更准确地描述用户的特性,同时具有良好的可扩展性.