A deadline-constrained scheduling for dynamic multi-instances parameter sweep workflow

摘要

Parameter sweep workflows have been adopted in scientific domains to orchestrate and automate parametric experiments studying the effects of different input values. Because of its independent parallel loop characteristic, a parameter sweep workflow can be accelerated by being dynamically executed as multiple parallel instances. When leveraging computing power provided on-demand by cloud computing, workflow scheduling is usually employed to manage the allocation of cloud resources in order to control usage cost and workflow execution time. However, most of the existing cloud workflow scheduling techniques are mainly designed for single workflow instance. Whereas, dynamic multi-instances execution negates the traversal of workflow graph to estimate cost and time used in most of the existing techniques. This paper proposes a new scheduling technique that is able to schedule parameter sweep workflow which is dynamically executed in multiple instances employing a new method to estimate the execution progress together with a workflow instantiation control and a cloud resource adjustment mechanism. The proposed technique, with the objective to minimize cost within a deadline, is evaluated using three existing task mapping heuristics. The simulation results show that the proposed technique is able to lower cloud usage cost when the time constraint is more relaxed. It also relies on the accuracy of the workflow progress estimation to achieve lower cost.