Cloud-to-cloud parallel data transfer via spawning intermediate nodes

摘要

This work proposes a parallel cloud-to-cloud data transfer method in which intermediate nodes are spawned and utilized in order to increase the transfer throughput by aggregating bandwidth. Existing data transfer methods do not take advantage of public cloud elasticity - the maximum throughput is limited by the available bandwidth of the source or/and destination transfer node. In this paper, a technique that does multi-part transfer by breaking down large files into smaller parts and transfer them in parallel via intermediate nodes is proposed. As intra DC bandwidth is often multiple times higher than inter DC or WAN, the proposed method allows transfer throughput higher than the bandwidth capacity of a single node. Based on modelling and experiment, the result shows that the proposed solution has potential to improve the overall performance.