An efficient replication scheme based on living-replicas estimation for distributed storage platforms

摘要

In the past few years, massive storage platforms are playing critical roles in many practical distributed systems (i.e., grid or cloud). To ensure desirable data availability, replication mechanism has been widely applied in those massive storage platforms. Meanwhile, a high level of data redundancy inevitably degrades the performance of storage management and bandwidth utilisation. In this paper, we propose an intelligent replication scheme which leverages failure pattern of storage nodes to estimate the actual number of living-replicas in a storage platform. In this way, the storage platform can make more accurate decisions on how many replicas should be generated for maintaining a given data availability. Extensive experiments based on real-world traces indicate that the proposed replication scheme can significantly reduce the overall data redundancy and maintain desirable data availability at the same time.