ATLAS grid workload on NDGF resources: Analysis, modeling, and workload generation

摘要

Evaluating new ideas for job scheduling or data transfer algorithms in large-scale grid systems is known to be notoriously challenging. Existing grid simulators expect to receive a realistic workload as an input. Such input is difficult to obtain in the absence of an in-depth study of representative grid workloads. In this work, we analyze the workload of the ATLAS experiment at CERN at the LHC, processed on the . resources of Nordic Data Grid Facility. ATLAS is one of the biggest grid technology users, with extreme demands for CPU power, data volume and bandwidth. The analysis is based on the data sample with ～ 1.6 million jobs, 3029 TB of data transfer, and 873 years of processor time. Our additional contributions are (a) scalable workload models that can be used to generate a synthetic workload for a given number of jobs, (b) an open-source workload generator software integrated with existing grid simulators, and (c) suggestions for grid system designers based on the insights of our analysis.