External Memory Algorithms: Dealing With Massive Data

摘要

The bottleneck in many applications that process massive amounts of data is the I/O communication between internal memory and external memory. The bottleneck is accentuated as processors get faster and parallel processors are used. The goal of this proposal is to deepen our understanding of the limits of I/O systems and massive data storage systems and to construct algorithms that are provably efficient. The three measures of performance are number of I/Os, disk storage space, and CPU time. Even when the data fit entirely in memory, communication can still be the bottleneck, and the related issues of caching become important.