Hands on with OpenMP4.5 and Unified Memory: Developing Applications for IBM's Hybrid CPU + GPU Systems (Part I)

摘要

High Performance Computing is steadily embracing heterogeneous systems for supporting a wide variety of workloads. Currently there are two main sources of heterogeneity in compute nodes: (a) different compute elements such as multicore CPUs, GPUs, FPGAs, etc. and (b) different types of memory including DDR, HBM, SSDs. Multiple compute elements and memory types present many opportunities for accelerating applications featuring stages characterized by different compute intensity, sequential or parallel execution, cache sensitivity, etc. At the same time programmers are facing multiple challenges in making necessary adaptations in their codes. In this study we employ IBM's OpenMP 4.5 implementation to program hybrid nodes with multiple CPUs and GPUs and manage on-node memories and application data. Through code samples we provide application developers with numerous options for memory management and data management. We consider simple functions using arrays and also complex and nested data structures.