Optimisation to the execution performance of grid job based on distributed file system

摘要

In grid computing, many applications are invoked by the means of job submission. Grid job submission comprises the following scenarios: file stage-in, execution and file stage-out. Most of the grid midware provides the job submission infrastructure to help to submit jobs, and GridFTP (file downloading) is used in the file stage operations. These scenarios exit mostly in batch mode. It causes computational resources waiting for the data transfer and limits the performance of the grid application, especially when the application is data/communication intensive. Using the file sharing of distributed file system instead of GridFTP in the job submission, the scenarios of grid job submission can be processed in parallel, whereas the waiting of computational resource can be eliminated. By this way, the execution performance of the grid job is promoted. After the comparison of typical distributed file systems, network attached storage that meets the requirements of grid is picked out, and the file sharing mode for the data exchange in grid job is proposed. In the tests of a series of data-communication-intensive jobs, the performance promotion is verified. Comparing with GridFTP, the average promotion with Common Internet File System is 16.8% to single-task jobs and 19.5% to multiple-task jobs.