Parallel K-Means Implementation for Data Clustering Using Hadoop Map-Reduce

摘要

The electronic information from online newspapers, journals, conference proceedings website pages and emails are growing rapidly which are generating huge amount of data. Controlling, indexing and searching of these huge electronic data is not feasible especially for human and alsofor search engines. Thus, automatic document organization is an important issue for this huge information. Using document clustering methods insights the data distribution or pre-process data for other applications. In this paper, a parallel clustering algorithm based on K-mean clusteringis proposed which is to iterate and optimize documents upload and access. Specifically, the proposed algorithm is implemented on Apache Hadoop architecture with the huge document access data set and the algorithm is evaluated on different conditions with different possible input documents.This paper is used to present the information of frequent access of each access and to suggest the pattern of document representation in the cloud storage.