Study of Association Rule Mining for Discovery of Frequent Item Sets on Big Data Sets

摘要

Frequent pattern mining is an significant research area in data mining. Since its introduction, it has drawn consideration of many researchers. Data generation and collection diagonally all areas enhance in size exponentially. Knowledge discovery and decision making requires the capability to process and extract imminent from "Big" Data in a scalable and efficient manner. Data mining is the application of sophisticated analysis to large amounts of data in order to discover new knowledge in the form of patterns, trends, and associations. With the beginning of the World Wide Web, the quantity of data stored and available by electronic means has developed extremely and the process of knowledge discovery (data mining) from this data has become very central for the business and scientific research communities. Many algorithms have been projected to mine frequent Item sets. Well-liked algorithms include level-wise Apriori based algorithms, tree based algorithms, and hyperlinked array structure based algorithms. While these algorithms are accepted and useful due to some good possessions, they also experience from some problems such as multiple database scans, recursive tree constructions, or multiple hyperlink adjustments. In the current era of big data, high volumes of a wide variety of valuable data of different veracities can be easily collected or generated at high velocity in a range of real-life applications. The problem of mining frequent queries in a relational database defined over a star schema is not easy even when we deal with only one table, because, on the one hand, the size of the search space is huge (because encompassing all possible queries that can be addressed to a given database), and on the other hand, testing whether two queries are equivalent (which entails redundant support computations) is NP-Complete. Therefore, the problem is even more difficult when they are applied to Big Data. In this paper we focus on handling high volumes of big data to generate frequent Item sets. In this paper, we examine in details the problems related to the Frequent Pattern Mining (FPM) in distributed and large data sets and present a general framework for adapting an exact Frequent Pattern Mining algorithm.