Cluster-based distributed dynamic cuckoo filter system for Redis

摘要

ABSTRACT With the exponential growth of network data storage scale, the issue of uniform distribution and efficient retrieval of data in the distributed storage systems such as the Redis cluster has received increasing attention in recent years. In view of the existing problems in scalability, usability and other aspects of the solution in current researches, we propose the distributed dynamic cuckoo filter system based on Redis cluster. On one hand, we introduce an efficient hash indexing structure–dynamic cuckoo filter (DCF), which only stores the fingerprint information of data, and has the automatically scalable capacity to meet the demand of data storage on a dynamic scale. On the other hand, we use an improved consistent hashing algorithm to construct Redis cluster and use the thorough communication mechanism of Redis cluster to achieve the data sharing and efficient utilisation of multi-machine filters. The scheme proposed in this paper can take the time and space efficiency into account, greatly promote the retrieval performance of massive data, and improve the reliability and availability of Redis cluster.