PReT: A Tool for Automatic Phase-Based Regression Testing

摘要

In this paper, we present our tool PReT, which performs automatic performance regression testing on software. PReT does non-intrusive profiling based on application snapshots to learn behaviour for performance regression tests and can identify any changes in the testing behaviour by comparing the current behaviour against a learned model. PReT annotates resource usage profiles with application stacktraces and uses a variation of k-means to learn the models per regression test online. On top of that, PReT uses version information of the software to identify change(s) that introduce(s) performance issue(s) if any. We show the usefulness of PReT in correctly identifying two real world performance bugs in Cassandra database server. We show that PReT is able to characterize the performance tests being run for the software with higher accuracy than a purely resource utilization based characterization technique.