Addressing Challenges in Visualizing Huge Call-Path Traces

摘要

Analysis and optimization of long-running applications on large-scale parallel systems is important to avoid unacceptable inefficiencies. Tracing is one of the most popular techniques for understanding the performance of parallel programs. Since tracing captures data in the time dimension, the size of a trace is linearly proportional to execution time. For that reason, traces of long-running executions of parallel programs may contain gigabytes or even terabytes of data. Presenting huge traces in a scalable fashion and identifying performance bottlenecks hidden in an ocean of data are challenging problems. To pinpoint performance bottlenecks effectively, a performance visualization tool needs to be relatively responsive and scalable. It also needs to be able to present both a global view of the performance of all threads and processes in a parallel execution, and a local view to see the full detail of a trace for an individual thread or process. Our approach to address this challenge is to use a client-server approach for trace visualization in hpctraceviewer, which is part of the HPC-TOOLKIT performance tools. This paper demonstrates the utility of our tool for identifying performance bottlenecks in large-scale executions through case studies with two Department of Energy procurement benchmarks: Algebraic Multi Grid (AMG) and Unstructured Mesh Transport (UMT) codes. Finally, the experiment shows that our implementation is scalable, rendering views of huge traces stored on remote supercomputers in a few seconds.