Dynamic file replica location and selection strategy in data grids

摘要

In this paper, we present the design of PU-DG optimizer toolbox (also known as PU-DG Optibox), which not only finds out the best strategy according to huge amount of simulation results but also proposes the min-max balancing workload method to upgrade the efficiency of execution in data grid environments. Data grid is one of key factors to build up large-scale dataset storage system and providing high performance computing capacity, by connecting scattered computing and storage resources located dispersedly in the grid. One major challenge in data grids is how to provide good and timely access to huge amount of data in distributed locations, given the high latency of interconnection networks. In this paper, we present the design framework of PU-DG Optibox for data grid environments. The proposed toolbox is a package containing a number of high-end techniques and methods running as middleware on top of data grid platforms, in order to optimize file downloads, by improving its efficiency and performance. The PU-DG Optibox provides users and developers possibilities for setting their own priority strategies. In addition, min-max balancing workload method is proposed to avoid that one computing node with lower network bandwidth to receive a job that has high complexity of job factor. Experimental results of techniques packaged in the proposed toolbox demonstrate its effectiveness.