Big Data Mining For Interesting Pattern Using MapReduced Technique

摘要

In the past few years, huge data are need to be stored, access and retrieved, that has increased drastically all over the world, this fast growth of data results in the need to analyse the huge amount of data. Due to lack of proper tools and programs, data remains unused and unutilized with important useful knowledge hidden. This study has carryout data mining interesting patterns in big data. Objectoriented design methodology was used. Frequent pattern growth algorithm on Hadoop using MapReduce has been used and particularly applied it to analyze maximum flight time in flight transaction data store of 108MB. MapReduce program consists of two functions Mapper and Reducer which runs on all machines in a Hadoop cluster. System was implemented in matlab. Computation has been performed to analyzed the actual flight time using user constraints, the constraints are arrival delay and actual elapse time. Airpeace carrier has the longest flight time, the analyzed carrier (Air peace) space was 20000x6 contained 712316 bytes. Thus, the execution time of the entire mining process was 1615 milliseconds.