Communication and Optimization Aspects on Hybrid Architectures

摘要

Most HPC systems are clusters of shared memory nodes. Parallel programming must combine the distributed memory paralleliza-tion on the node inter-connect with the shared memory parallelization inside of each node. The hybrid MPI+OpenMP programming model is compared with pure MPI and compiler based parallelization. The paper focuses on bandwidth and latency aspects, but also whether programming paradigms can separate the optimization of communication and computation. Benchmark results are presented for hybrid and pure MPI communication.