Way of Measuring Data Transfer Delays among Graphics Processing Units at Different Nodes of a Computer Cluster

摘要

Abstract The basics of load tests for a computer cluster with a large number of GPUs (graphics processing units) distributed over the cluster’s nodes are presented and implemented as a program code. Information about the time delays in the transfer of data of different sizes among all GPUs in the system is collected as a result. Two modes of tests, ‘‘all to all’’ and ‘‘one to one,’’ are developed. In the first mode, all GPUs transfer data to all GPUs simultaneously. In the second mode, only the transfer between two GPUs proceeds at a single moment in time. Using test results obtained on the K60 computer cluster at the Keldysh Institute of Applied Mathematics, Russian Academy of Sciences, it is shown that the interconnector medium of the supercomputer is inhomogeneous in data transfer among the GPUs not only for transfer through the network, but also for the GPUs in a common node of the computer cluster.