Maximizing the GPU resource usage by reordering concurrent kernels submission

摘要

The increasing amount of resources available on current GPUs sparked new interest in theproblem of sharing its resources by different kernels. While new generations of GPUs supportconcurrent kernel execution, their scheduling decisions are taken by the hardware at runtime. Thehardware decisions, however, heavily depend on the order at which the kernels are submitted toexecution. In this work, we propose a novel optimization approach to reorder the kernels invocationfocusing on maximizing the resources utilization, improving the average turnaround time.We model the kernel assignments to the hardware resources as a series of knapsack problemsand use a dynamic programming approach to solve them.We evaluate our method using kernelswith different sizes and resource requirements. Our results show significant gains in the averageturnaround time and system throughput compared to the kernels submission implemented inmodern GPUs.