A data-intensive workflow scheduling algorithm for large-scale cooperative work platform

摘要

With the development of the society and the advancement of technology, the collaboration is being more and more important. A large-scale cooperative work platform is a platform which integrates computational, storage and network resources distributed in various organizations or locations and utilize these resources cooperatively to achieve one goal, such as an e-science or e-business platform. The data-intensive workflow on these platforms has gained much more attentions in recent times. Data-intensive workflow needs to access, process and transfer large datasets that may each be replicated on different data hosts. In this paper, we introduce an algorithm MDTT to select the resource set which the task should be mapped. Our experiments show that our algorithm is able to minimize the total makespan of data-intensive workflow and the time of data transferring.