A new scalable distributed k-means algorithm based on Cloud micro-services for High-performance computing

摘要

The paper aims to propose a distributed clustering method for High performance computing (HPC) models and, its application for medical image processing. The communication cost is one of the great challenges, which minimizes the scalability of parallel and distributed computing models. Indeed, it reduces significantly the performance of HPC systems where these models are assigned to be implemented. In this paper, we present a new distributed k-means method which integrates virtual parallel distributed computing model with a low communication cost mechanism. The k-means method is performed as a distributed service within a cooperative micro-services team which uses asynchronous communication mechanism based on AMQP protocol. We design and implement a parallel and distributed HPC application for MRI image segmentation assigned to be deployed on cloud. Experimental results show that the proposed method (DSCM) and its assigned model reach high degree of scalability. We expect this clustering approach to provide scalable HPC applications for big data clustering.