Apriori algorithm optimization based on Spark platform under big data

摘要

To extract useful information from massive data, based on the Spark platform, related techniques of the recommended algorithm were studied. Based on experimental data of a certain scale, the relationship between the various influencing factors of moral education evaluation was discussed and applied to the ranking statistics and correlation analysis functions. The evaluation index system of moral education was obtained. The results showed that Spark performed better than Hadoop in the parallelization implementation of the recommended algorithm. In the case of heterogeneous Spark clusters, the HSATS adaptive task scheduling strategy reduced the completion time of the job, and the utilization of cluster node resources was more reasonable. Therefore, the proposed optimization scheme of the recommendation algorithm improves the evaluation index of the recommendation system.