CloudFinder: A System for Processing Big Data Workloads on Volunteered Federated Clouds

摘要

The proliferation of private clouds that are often underutilized and the tremendous computational potential of these clouds when combined has recently brought forth the idea of volunteer cloud computing (VCC), a computing model where cloud owners contribute underutilized computing and/or storage resources on their clouds to support the execution of applications of other members in the community. This model is particularly suitable to solve big data scientific problems. Scientists in data-intensive scientific fields increasingly recognize that sharing volunteered resources from several clouds is a cost-effective alternative to solve many complex, data- and/or compute-intensive science problems. Despite the promise of the idea of VCC, it still remains at the vision stage at best. Challenges include the heterogeneity and autonomy of member clouds, access control and security, complex inter-cloud virtual machine scheduling, etc. In this paper, we present CloudFinder, a system that supports the efficient execution of big data workloads on volunteered federated clouds (VFCs). Our evaluation of the system indicates that VFCs are a promising cost-effective approach to enable big data science.