Active Global Address Space: A Global Memory Model for Adaptive Extreme-Scale Execution

摘要

As the leading edge of high-performance computing advances forward, supercomputers continue to increase rapidly in scale and complexity. To support the evolving architectural trends, an emerging class of runtime systems are centered around the asynchronous, massively multi-threaded model of computation in which lightweight threads operate on data residing in globally shared memory. Global shared memory has well-proven benefits as it provides an abstraction of unified, contiguous memory space on top of distributed memory hardware. Prior approaches to global memory, however, have either been purely static (e.g. partitioned global address space) or plagued by performance issues related to granularity and coherence (e.g. distributed shared memory). Furthermore, maintaining a scalable high- performance virtual global address space using distributed memory hardware has proven to be challenging, particularly in the case of migrating data and dynamic system configurations.;This thesis implements an active global address space (AGAS) where data residing in memory can actively migrate from one node to another while retaining its global address. The virtualization of memory resource in the global memory hierarchy provides several key capabilities to the runtime system. Most important of them all, transparent migration of global data, allows the runtime system to optimize locality of execution and remain resilient against faults. This thesis explores several possible models and implementations of global virtual memory in the form of AGAS---both in software and hardware---and develops algorithms and criteria for correctness of execution in presence of concurrently migrating data. In particular, the software implementations make use of concurrent lock-free data structures and lazy cache-coherence protocols to achieve high performance, whereas techniques such as reference counting and message forwarding ensure safe access to migrating data. Hardware-assisted global addressing will play a crucial role in next-generation HPC systems. To that end, this thesis demonstrates a novel scheme to implement AGAS in the network fabric by offloading global addressing to network switches. These ideas have been implemented in the context of High Performance ParalleX (HPX-5), a novel runtime system suited for massive-scale parallel execution.;As the dynamic execution capability afforded by AGAS comes at a potential price, one of the important goals of this thesis is to quantify the overhead and opportunity cost of dynamic data and contrast it against the benefits offered by load balancing. We analyze several HPX-5 applications to characterize global data access patterns and understand the nature of load imbalance. Finally, we develop a dynamic load balancing framework in HPX- 5 that relies on novel schemes to automatically migrate data in AGAS, and demonstrate that it helps improve performance of certain applications by up to 30%.