Design and implementation of a MPI-based parallel file system

摘要

This paper presents the design of an MPI (Message Passing Interface)-based parallel file system, MPFS. MPI-IO is an extension of MPI which supports flexible logical file partition and physical file organization as well as a rich set of file access functions. MPFS enables users to specify both logical file partitions among user processes and physical file data layouts across data servers. The present implementation includes full support of file data distribution, asynchronous I/O, shared file pointers, collective I /O operations, and limited support of data layout hints. Moreover, MPFS achieves portability through a user library that conforms to a generic UNIX interface. However, MPFS does not support the FORTRAN interface, error handling or profiling in the MPI-IO specification. MPFS has been implemented on a workstation cluster connected by Fast Ethernet. The performance measurements of MPFS show that the message passing overhead is minimal, and that the performance is limited by the bandwidth of Fast Ethernet.