Dynamic Co-allocation Scheme for Parallel Data Transfer in Grid Environment

摘要

The large sized data sets are replicated in more than one site for the better availability to the nodes in a grid. Downloading the dataset from these replicated locations have practical difficulties, due to network traffic, congestion, frequent change-in performance of the servers, etc. In order to speed up the download, complex server selection techniques, network and server loads are used. However, consistent performance is not guaranteed due to the shared nature of network links of the load on them, which can vary unpredictably. Hence, we find interest in a co-allocated download model, which enables parallel download of replicated data from multiple servers. In this paper, we proposed a dynamic co-allocation scheme for parallel data transfer in grid environment, which copes up with highly inconsistent network performances of the servers. We have developed an algorithm using circular queue, with which, the data transfer tasks are allocated onto the servers in duplication. Our scheme is highly fault tolerant one, in other words, the process of data transfer will neither be interrupted nor paralyzed, even when the link to servers under consideration is broken or idleness of the servers, whereas, none of the existing mechanisms consider the situation. We used Globus toolkit for our framework and utilized the partial copy feature of GridFTP. We compared our scheme with the existing schemes and the preliminary results show notable improvement in overall completion time of data transfer.