Performance analysis of distributed symmetric sparse matrix vector multiplication algorithm for multi-core architectures

摘要

Sparse matrix vector multiply (SpMVM) is an important kernel that frequently arises in high performancerncomputing applications. Due to its low arithmetic intensity, several approaches have been proposed in literaturernto improve its scalability and efficiency in large scale computations. In this paper, our target systemsrnare high end multi-core architectures and we use messaging passing interface + open multiprocessing hybridrnprogramming model for parallelism. We analyze the performance of recently proposed implementation ofrnthe distributed symmetric SpMVM, originally developed for large sparse symmetric matrices arising in abrninitio nuclear structure calculations. We study important features of this implementation and compare withrnpreviously reported implementations that do not exploit underlying symmetry. Our SpMVM implementationsrnleverage the hybrid paradigm to efficiently overlap expensive communications with computations. Ourrnmain comparison criterion is the ‘CPU core hours’ metric, which is the main measure of resource usagernon supercomputers. We analyze the effects of topology-aware mapping heuristic using simplified networkrnload model. We have tested the different SpMVM implementations on two large clusters with 3D Torusrnand Dragonfly topology. Our results show that the distributed SpMVM implementation that exploits matrixrnsymmetry and hides communication yields the best value for the ‘CPU core hours’ metric and significantlyrnreduces data movement overheads.