Spark-based Parallel Collaborative Filtering Recommendation Algorithm

摘要

The rapid development of Internet information technology makes the problem of information overload become more and more serious, and recommendation system is one of the effective ways to solve this problem which is favored by people. However, for the massive data information, the recommended algorithm faces the bottleneck problem of processing speed and computing resources, so this paper proposed a parallel collaborative filtering recommendation algorithm based on Spark. The RLPSO algorithm is used to optimize the clustering factor of the K-means clustering algorithm by associating users with similar interests into a cluster and using the recommended algorithm for users to recommend is implemented on the Spark platform. The experimental results show that the improved algorithm has a significant improvement in the prediction accuracy, and has a higher speedup and stability compared with the traditional collaborative filtering recommendation algorithm.