Cluster file system with metadata server for storage of parallel log structured file system metadata for a shared file

摘要

Data from a group of distributed processes to a shared file is written using a parallel log-structured file system. A metadata server of a cluster file system is configured to communicate with a plurality of object storage servers of the cluster file system over a network. The metadata server further configured to implement a Parallel Log Structured File System (PLFS) library to coordinate storage on one or more of the plurality of object storage servers of a plurality of portions of a shared file generated by a plurality of applications executing on compute nodes of the cluster file system and to store metadata for the plurality of portions of the shared file. Concurrent writes to the shared file are decoupled by writing the plurality of portions of the shared file generated by each of the plurality of applications to independent write streams for each application. The metadata server communicates with a plurality of applications executing on the compute nodes over the network to process metadata requests from the applications.