A Parallel Association Mining Algorithm for Analyzing Passenger Travel Characteristics

摘要

#$%^&*AU2020101071A420200723.pdf#####Abstract Frequent pattern mining is an effective approach for spatiotemporal association analysis of mobile trajectory big data in data-driven intelligent transportation systems, which can provide decision making for urban transportation optimization and control of smart cities. While existing parallel algorithms have been successfully applied to frequent pattern mining of large-scale trajectory data on Hadoop employing MapReduce, two major challenges are how to overcome the inherent defects of Hadoop to cope with taxi trajectory big data including massive small files and how to discover the implicitly spatiotemporal frequent patterns with the MapReduce paradigm. To conquer these challenges, this invention presents a MapReduce-based Parallel Frequent Pattern growth algorithm, MR-PEP, to analyze the spatiotemporal characteristics of passenger travel using large-scale taxi trajectories with massive small file processing strategies on a Hadoop platform. More specifically, the present invention mainly consists of three phases. We first implement three methods, i.e., Hadoop Archives (HAR), CombineFilenputFormat (CFIF) and Sequence Files (SF), to overcome the existing defects of Hadoop, and then propose two strategies based on their performance evaluations in terms of memory consumption and execution efficiency. Next, we incorporate SF into Frequent Pattern growth (FP-growth) algorithm, and then implement the optimized FP-growth algorithm on a MapReduce framework. Finally, we analyze the characteristics of passenger travel in both spatial and temporal dimensions by MR-PFP in parallel. The present invention has broad application in big data analytics.