Modeling Impact of Execution Strategies on Resource Utilization

摘要

The analysis of the hundreds of petabytes of raw and derived HEP (High Energy Physics) data will necessitate exascale computing. In addition to unprecedented volume, these data are distributed over hundreds of computing centers. In response to these application requirement, as well as performance requirement by using parallel processing (i.e., parallelism), and as a consequence of technology trends, there has been an increase in the uptake of supercomputers by HEP projects. The increased prevalence of supercomputers has important implications in the execution strategy employed. Because each resource is now a significant workhorse, the importance and sensitivity of correct resource selection for high-utilization and good response time are higher. As the task types using resources of high-throughput computing and high-performance computing by HEP projects become more varied (e.g., reconstruction, simulation, digitization, etc.), better models for generalized resource selection for a wide range of heterogeneous tasks are needed, thus to respond to different heterogeneous workloads. Another important consideration is the selection of resources that are likely to facilitate the utilization of the available hours in a meaningful time window. For example, there is no pointing selecting resources that might enable faster execution time but will not support the appropriate number of core-hours in the time window of interest. Thus, any accurate resource selection strategy must also consider the implications of the load of the supercomputer chosen. In this short paper, we will study the interplay of the load on the supercomputer with the tasks placed on the supercomputer.