Network-Aware Distributed Storage Cache for Data Intensive Environments

摘要

Modern scientific computing involves organizing, moving, visualizing, and211u001eanalyzing massive amounts of data at multiple sites around the world. The 211u001etechnologies, the middleware services, and the architectures that are used to 211u001ebuild useful high-speed, wide area distributed systems, constitute the field of 211u001edata intensive computing. In this paper the authors describe an architecture for 211u001edata intensive applications where they use a high-speed distributed data cache as 211u001ea common element for all of the sources and sinks of data. This cache-based 211u001eapproach provides standard interfaces to a large, application-oriented, 211u001edistributed, on-line, transient storage system. They describe their 211u001eimplementation of this cache, how they have made it network aware, and how they 211u001edo dynamic load balancing based on the current network conditions. They also show 211u001elarge increases in application throughput by access to knowledge of the network 211u001econditions.