The difficulty of handling out-of core data limits the performance of supercomputers as well as the potential of the parallel machines. Since writing an efficient out--of core version of a program is a difficult task and virtual memory systems do not perform well on scientific computations, we believe that there is a clear need for compiler directed explicit I/O approach for out-of core computations. In this paper, we first present an out-of-core compilation strategy based on a disk storage abstraction. Then, we offer a compiler algorithm to optimize locality of disk accesses in out--of core codes by choosing a good combination of file layouts on disks and loop transforma- tions. We introduce memory coefficient and processor coefficient concepts to characterize the behavior of out--of core programs under different memory constraints. We also enhance our algorithm to handle data-parallel programs which contain multiple loop nest. Our initial experi- mental results obtained on IBM SP--2 and Intel Paragon provide encouraging evidence that our approach is successful at optimizing programs which depend on disk--resident data in distributed- memory machines.
展开▼
