Jcluster: an efficient Java parallel environment on a large-scale heterogeneous cluster

摘要

In this paper, we present Jcluster, an efficient Java parallel environment that provides some critical services, in particular automatic load balancing and high-performance communication, for developing parallel applications in Java on a large-scale heterogeneous cluster. In the Jcluster environment, we implement a task scheduler based on a transitive random stealing (TRS) algorithm. Performance evaluations show that the scheduler based on TRS can make any idle node obtain a task from another node with much fewer stealing times than random stealing (RS), which is a well-known dynamic load-balancing algorithm, on a large-scale cluster. In the performance aspects of communication, with the method of asynchronously multithreaded transmission, we implement a high-performance PVM-like and MPI-like message-passing interface in pure Java. The evaluation of the communication performance is conducted among the Jcluster environment, LAM-MPI and mpiJava on LAM-MPI based on the Java Grande Forum's pingpong benchmark.