An Efficient Fault Tolerance Framework for Distributed In-memory Caching Systems

摘要

With the development of the information age, many large database applications have introduced distributed in-memory object caching systems, of which Memcached is one of the most typical. However, Memcached does not have fault-tolerant capabilities. In order to make Memcached enable fault tolerance, Cocytus introduced Reed-Solomon codes and distributed protocols into Memcached. Cocytus saves significant memory compared to primary-backup replication when tolerating the same number of failures. However, the relatively complex finite-field calculations used by RS codes and the high network transmission cost during data reconstruction are becoming new bottlenecks. This paper introduces RDP codes into distributed Memcached to optimize the calculation performance in Cocytus. In addition, this paper adopts RDOR scheme and Collective Reconstruction Read to speed up the data reconstruction. Compared with Cocytus, which uses RS codes for fault tolerance, the new distributed Memcached with 4 data nodes and 2 check parity nodes reduces reconstruction overhead by up to 31%.