Efficient Parallel Data Mining for Massive Datasets: Parallel Random Forests Classifier

摘要

Data mining refers to the process of finding hidden patterns inside a large dataset. While improving the accuracy of those algorithms has been the main focus of past research, massive dataset size imposes another challenge. Parallel and distributed processing techniques have been applied to data mining algorithms to make them scalable. In this paper, we discuss a new emerging data mining algorithm, random forests, and its parallelization based on VCluster, a portable parallel runtime system we have developed for a cluster of multiprocessors. Random forests is an ensemble of many decision trees and the classification is performed by majority voting by those decision trees. We also present the experimental results on the performance of parallel random forests approach.