Implementation of collective I/O in the Intel Paragon Parallel File System: initial experiences

摘要

A majority of parallel applications achieve parllelism by partitioning data over multipele processors. Accessing distributed data structures such as arrays from files often requires each prcessor to make a large number of small non-contiguous data requets. this problem can be addressed by replacing small non-contiguous requests by large collertive requests. This approach, knonw as collective I/I. has been found to work extremely wel in practice. HTis paper descibes implementation and evaluation of a collective I/O prototype in the Paralel File system of the Intel paragon operating system. we evaluate the collective I/O performance using its comparison with the PFS M_RECORD and M_UnIX I/O modes. It is observed that collective I/O provides significant profrmance improvement over accesses in the M_MNIX mode. However, in many cases, various implementation overheads cause collective I/I to provide lower performance than the M_RECORD I/O mode.