Parallel Data Analysis and its Implementation on PC Cluster

摘要

There are numerous datasets downloaded or extracted easily from the Internet with/without purpose. The number of their records are so huge that we have to seek how to classify them when we want to extract useful information from them. Many computer-intensive data analysis methods have been developed. Some of them, however, are not focused on execution for huge datasets. In short, they tend to waste much time.rnWe have studied making typical data analysis methods in parallel and sought for parallel oriented data analysis, using PC cluster. It gives us enough computing power, but to utilize it, we have to solve the problem, "How can we adapt well-known statistical methods into PC cluster?"rnIn this study, we focus on one of the well-known non-hierarchical clustering method k-means and report how to implement it into parallel environment properly. We have already reported κ-means parallel execution in PVM environment, however, another PC cluster library MPI is more popular and suitable for κ-means. Through numerical examples, we show its effectiveness and offer some viewpoints for parallel oriented statistical analysis.