Analyzing Testing Strategy Tradeoffs to Improve Systems Engineering Effectiveness and Efficiency

摘要

Measurement-driven system development focuses on using quantitative data to evaluate capabilities, benefits, progress, and tradeoffs as well as identify improvement opportunities. This paper describes a controlled study that addresses software testing effectiveness and focuses on the combination of individual testing techniques into team-based testing strategies. This analysis is intended to enable measurement-driven process improvement by characterizing how testing effectiveness relates to several factors, including testing strategy, software type, and developer expertise. In this study, a representative group of software development professionals applied common testing techniques to different types of software. This study compares the six possible team combinations of three testing techniques: (1) code reading by stepwise abstraction, (2) functional testing using equivalence partitioning and boundary value analysis, and (3) structural testing using 100% statement coverage criteria. Thirty-two professional developers applied the techniques to three unit-sized programs in a fractional factorial experimental design. The major results of this study are the following.