A Practical Data-driven Framework for Parallel Data Mining

摘要

In many practical applications, data mining results must be quickly delivered. To achieve the required efficiency, without sacrificing the quality of the results, practitioners are now looking at ways to parallelize the most computationally expensive steps of the data mining process. Realizing mat a complete rewriting of existing sequential programs into parallel ones is often too tedious and expensive, we propose a framework which re-uses existing sequential programs to perform parallel data mining on a computer cluster. The proposed framework relies on the JavaParty system and can be used to parallelize both Java and non-Java programs. This paper details the framework, illustrates the implementation, and presents early experimental results showing the benefits of the approach.