A hybrid disaster-tolerant model with DDF technology for MooseFS open-source distributed file system

摘要

In a distributed file system (DFS), a disaster-tolerant model provides high data security in the data disaster event by geographically isolates the backup data, while the hot-backup model provides high availability when the main server is offline. Therefore, the combined model is widely concerned in multiple fields. However, previous investigations have neglected to the disaster tolerance for open-source distributed file systems. When a server run into a fault status and performs backup switch, the remote data center cannot receive any data replicas. We proposed a hybrid model in this work to provide disaster-tolerance especially for open-source DFS by build geographically isolate replicas. In addition, we optimize the model with direct data fetch (DDF) technology. DDF can bypass the main server, and retrieve data directly from the data node. The model was implemented on an open-source DFS named MooseFS. Compared with HDFS and IBM/PPRC, the hybrid model with DDF provides a strong performance boost to the hybrid model during the switch procedure, and this hybrid model performs best for storage of large amount of files in small size.