The I-Cluster Cloud: distributed management of idle resources for intense computing

摘要

I-Cluster is an HP Laboratories Grenoble initiative, in collaboration with the ID-IMAG laboratory of INRIA Rhone-Alpes, HP Brazil and the Catholic University of Rio Grande do Sul (PUCRS). I-Cluster consists of a framework of tools that transparently takes advantage of unused network resources and federates them, in order to crystallize into specific virtual functions such as supercomputing. By doing this, I-Cluster enables automatic real-time analysis of the availability and workload of machines on an intranet. When the instantiation of a supercomputing function is requested by a user, I-Cluster determines the most appropriate set of machines for carrying out this function, allocates the machines into a virtual cluster and proceeds with the execution of the function. In order to address security issues, I-Cluster uses an "execution sandbox" on each machine of the intranet, which is transparent to the user and enables the use of local computing resources at idle periods, while securely protecting the local user data and jobs. In this paper we introduce the I-Cluster initiative and its overall architecture and present the main issues addressed in the conception of the I-Cluster framework, such as solving peer-to-peer computing security issues using OS sandboxing, self-organization and resilience to unanticipated disconnections of large and heterogeneous community of computers, as well as automatic resource collection. To validate the I-Cluster framework we both present experimental results obtained with a small scale prototype and simulated results for environments with a larger number of resources.