Tridiagonalization of a dense symmetric matrix on multiple GPUs and its application to symmetric eigenvalue problems

摘要

For software to fully exploit the computing power of emerging heterogeneous computers, not only must thernrequired computational kernels be optimized for the specific hardware architectures but also an effectivernscheduling scheme is needed to utilize the available heterogeneous computational units and to hide the communicationrnbetween them. As a case study, we develop a static scheduling scheme for the tridiagonalizationrnof a symmetric dense matrix on multicore CPUs with multiple graphics processing units (GPUs) on a singlerncompute node.We then parallelize and optimize the Basic Linear Algebra Subroutines (BLAS)-2 symmetricrnmatrix-vector multiplication, and the BLAS-3 low rank symmetric matrix updates on the GPUs.We demonstraternthe good scalability of these multi-GPU BLAS kernels and the effectiveness of our scheduling schemernon twelve Intel Xeon processors and three NVIDIA GPUs. We then integrate our hybrid CPU-GPU kernelrninto computational kernels at higher-levels of software stacks, that is, a shared-memory dense eigensolverrnand a distributed-memory sparse eigensolver. Our experimental results show that our kernels greatly improvernthe performance of these higher-level kernels, not only reducing the solution time but also enabling the solutionrnof larger-scale problems. Because such symmetric eigenvalue problems arise in many scientific andrnengineering simulations, our kernels could potentially lead to new scientific discoveries. Furthermore, theserndense linear algebra algorithms present algorithmic characteristics that can be found in other algorithms.rnHence, they are not only important computational kernels on their own but also useful testbeds to study thernperformance of the emerging computers and the effects of the various optimization techniques.