Optimal metadata replications and request balancing strategy on cloud data centers

摘要

In large-scale cloud data centers, metadata accesses will very likely become a severe performance bottleneck as metadata-based transactions account for over 50% of all file system operations. Clusters of Metadata Servers (MDS) that provide metadata searching service can improve the system performance significantly. For a data stored in cloud data centers, there may be several MDS storing the metadata replicas. Therefore, when a data request arrives, it has many potential metadata paths, one of which shall be chosen to obtain the best performance. In this paper, we attempt to determine the number of MDS that each data object in the system shall have and the request rates that each MDS shall serve, in order to achieve the minimum mean response time (MRT) of all the metadata requests. The target optimal constrained function has been formulated and a novel metadata request balancing algorithm based on request arrival rates has been proposed, which can find near-optimal solutions by a theoretical proof. In our experiments, we compare our algorithm with widely used hashing functions that have 0, 1, 2, 3 replicas, respectively. We validate our findings via simulations with respect to several influencing factors and prove that our proposed strategy is scalable, flexible and efficient for the real-life applications. Some interesting perspectives of the work are also presented at the end of this paper.