LAR: Locality-Aware Reconstruction for erasure-coded distributed storage systems

摘要

Many modern distributed storage systems adopt erasure coding to protect data from frequent server failures for cost reason. Reconstructing data in failed servers efficiently is vital to these erasure-coded storage systems. To this end, tree-structured reconstruction mechanisms where blocks are transmitted and combined through a reconstruction tree have been proposed. However, existing tree-structured reconstruction mechanisms build reconstruction trees from the perspective of available network bandwidths between servers, which are fluctuating and difficult to measure. Besides, these reconstruction mechanisms cannot reduce data transmission. In this study, we overcome these limitations by proposing LAR, a locality-aware tree-structured reconstruction mechanism. LAR builds reconstruction trees from the perspective of data locality, which is stable and easy to obtain.More importantly, by building reconstruction trees that combine blocks closer to each other first, LAR can reduce the data transmitted through the network core and hence speed up reconstruction.We prove that a minimum spanning tree is an optimal reconstruction tree that minimizes core bandwidth usage.We also design and implement a general reconstruction framework that supports all tree-structured reconstruction mechanisms and nearly all erasure codes. Large-scale simulationsoncommonlydeployednetwork topologiesshow that LAR consumes 20%–61% less core bandwidth than previous reconstruction mechanisms. Thorough experiments on a testbed consisting of 40 physical servers show that LAR improves proactive recovery throughput by 23% at least and improves degraded read rate by up to 68%.