Parallel I/O for Scientific Applications on Heterogeneous Clusters: A Resource-utilization Approach

摘要

Clusters of workstations and PCs are gaining more popularity as an economical resource for parallel computing. However, clusters are prone to be heterogeneous in I/O as well as in processing power. As a result, the performance of I/O intensive parallel scientific applications in which processes are closely synchronized together can be very poor, due to a bottleneck caused by the placement of one or more I/O servers without appropriate consideration of I/O resources available. This paper presents heuristics to choose the number of I/O servers and place them on physical processors in a way to better exploit disk I/O resources and minimize total I/O time in heterogeneous cluster environments. Our experimental results show that the heuristics improve overall parallel I/O performance as much as by a factor of 2-4 on a cluster of SMPs, compared with the simple placements typically used in runtime I/O libraries and applications. Our approach is also suitable for other parallel I/O libraries, to better support I/O intensive parallel scientific applications in cluster environments.