基于垂直FP树的并行频繁项集挖掘

摘要

随着数据量的快速增长、数据存储的分散化程度不断提高,对并行分布式数据挖掘算法的需求越来越迫切.文章提出了一种基于垂直FP树的分布式频繁项集挖掘算法DVFP.DVFP采用一种称为垂直FP树(VFP)的格式来存放数据,并同时采用数据并行和任务并行的策略.文章还提出了一种新的序列化方法来对VFP树进行编码,大大减少了处理节点间的通信开销.实验验证DVFP算法在灵活性和处理时间上与现有的分布式算法相比具有较大优势.%With the rapid growth of the amount of distributed data, the need for parallel and distributed data mining algorithm becomes more and more pressing. This paper presents a distributed algorithm based on vertical FP-tree called DVFP for mining frequent item. DVFP u-ses a data struct called vertical FP tree (VFP) to store the data, and data parallel and task parallel strategy are used at the same time. This paper also presents a new method to serialize VFP, which greatly reducing the time of communication. Experiments shows that DVFP algorithm has a larger advantage in flexibility and processing time with existing distributed algorithm.