Network optimization for distributed memory file system on high performance computers

摘要

On high performance computers, each compute node's demand for memory may be different. Building a distributed file system in idle memory can improve the memory utilization of the whole system. When memory takes place of disk, the Socket-based communication becomes the main bottleneck. As most of current high performance computers support RDMA, RBP (RDMA Buffer Pool) was referred to optimize the network performance of such an in-memory file system. MooseFS was adopted by RBP and deployed in TH-1A supercomputer. The experiment results showed that the RBP-based method could improve the speed of clients and the aggregate bandwidth of servers for sequential read and write significantly. For a single client, it increased the speeds of sequential read and write by a factor of 2.0～2.6. For a single server, it increased the collective bandwidths of sequential read and write by a factor of 2.0～2.4.