Coverage Specification for Test Case Intent Preservation in Regression Suites

摘要

Regression testing ensures that previous faults do not recur. When a fault is reported and fixed, the testing team augments the test suite with a new test case that exercises the fault in the original program. Typically the new test case covers patterns of program elements associated with the fault. However, this test might become obsolete (i.e., does not cover the targeted patterns anymore) when other code modifications are made. Nevertheless, current coverage metrics might still report acceptable values thus misleading the testing team. In this paper, we present a coverage specification language and a methodology to preserve the intent of test cases in a regression test suite. We implemented our methodology for C programs and for def-use program elements. With our method, a) a coverage specification language enables testers to specify def-use pairs and associate them with tests, b) the def-use pairs' specifications recognize the pairs across subsequent versions of the program, and c) the tool computes run time assembly level addresses for def-use pairs and guarantees intent preservation. Preliminary results show that our method works for program changes involving insertions and deletions of several lines of code.