HDFS framework for efficient frequent itemset mining using MapReduce

摘要

Association rule mining is a very essential data mining technique in different fields. The enormous development of the information needs increased computational power. To address this issue, it is important to study executions of mining algorithms. To find out the frequent itemsets is an essential and vital issue in numerous information mining applications. There are many algorithms present to extract frequent itemsets like Apriori and FP-Growth. But these algorithms lack properties like parallelization, load balancing, data distribution, and fault tolerance on large clusters or big data. A Modified Apriori method is introduced here in which, the mappers and reducers will work simultaneously. This method uses three MapReduce to calculate frequent itemset. The third MapReduce is used to decompose itemsets and gives the final result. In this paper a new scheme or algorithm is proposed that will reduce the execution time for the massive database and works efficiently on number of nodes by using Modified Apriori algorithm.