Localizing software performance regressions in web applications by comparing execution timelines

摘要

A performance regression in software is defined as an increase in an application step's response time as a result of code changes. Detecting such regressions can be done using profiling tools; however,investigatingtheir root cause is a mostly-manual and time-consuming task. This statement holds true especially when comparingexecution timelines, which are dynamic function call trees augmented with response time data; these timelines are compared to find theperformance regression-causes- the lowest-level function calls that regressed during execution. When done manually, these comparisons often require the investigator to analyze thousands of function call nodes. Further, performing these comparisons onweb applicationsis challenging due to JavaScript's asynchronous and event-driven model, which introduce noise in the timelines. In response, we propose a design -Zam - that automatically compares execution timelines collected from web applications, to identify performance regression-causes. Our approach uses a hybrid node matching algorithm that recursively attempts to find the longest common subsequence in each call tree level, then aggregates multiple comparisons' results to eliminate noise. Our evaluation ofZamon 10 web applications indicates that it can identify performance regression-causes with a path recall of 100% and a path precision of 96%, while performing comparisons in under a minute on average. We also demonstrate the real-world applicability ofZam, which has been used to successfully complete performance investigations by the performance and reliability team in SAP.