A Compiler Driven Out-of-Core Programming Approach for Optimizing Data Locality in Loop Nests

摘要

Most scientific programs have large input and output data sets that require out-of-core programming or the use of virtual memory management (VMM). Often, VMM is not an effective approach because it results frequently in substantial performance reduction. In contrast, compiler driven I/O management allows a program's data sets to be retrieved explicitly in parts, called blocks or tiles. In this paper, we offer an out-of-core programming schema to optimize locality of disk accesses in stepped problems by choosing a good combination of data tiling and loop transformations. The experimental results provide strong evidence that an out-of-core programming approach combined with non-standard data mappings and corresponding loop transformations can improve the performance of out-of-core problems by one order of magnitude or more.