Multi-dimensional batch scheduling framework for high-end supercomputers.

摘要

In the field of high performance computing (HPC), batch scheduling plays a critical role. They determine when and how to process the various jobs waiting for service. Conventional batch schedulers allocate user jobs solely based on their CPU footprints. However, for a given user job, it requires many different resources during its execution, such as power, network, I/O bandwidth, etc. Today's job schedulers rarely take into account these resource requirements which sometimes turn out to be the Achilles' heel of system-wide performance. In this research, we propose a multi-dimensional batch scheduling framework for high-end supercomputers. Our research aims to treat these common but often ignored resources (e.g., power, network, bandwidth) as schedulable resource and further transform each scheduling into a multi-objective optimization process. Our main contributions consist of a set of scheduling models and policies, aiming at addressing the issues in batch scheduling for large-scale production supercomputers. We evaluate our design by means of trace-based simulations using real workload and performance traces from production systems. Experimental results show our methods can effectively improve batch scheduling regarding user satisfaction, system performance and operating cost.