Efficiency of Software Testing Techniques: A Controlled Experiment Replication and Network Meta-analysis

摘要

Background. Common approaches to software verification include static testing techniques,such as code reading, and dynamic testing techniques, such as black-box and white-box testing.Objective. With the aim of gaining a better understanding of software testing techniques,a controlled experiment replication and the synthesis of previous experiments which examine theefficiency of code reading, black-box and white-box testing techniques were conducted. Method.The replication reported here is composed of four experiments in which instrumented programs wereused. Participants randomly applied one of the techniques to one of the instrumented programs.The outcomes were synthesized with seven experiments using the method of network meta-analysis(NMA). Results. No significant differences in the efficiency of the techniques were observed.However, it was discovered the instrumented programs had a significant effect on the efficiency. TheNMA results suggest that the black-box and white-box techniques behave alike; and the efficiencyof code reading seems to be sensitive to other factors. Conclusion. Taking into account thesefindings, the Authors suggest that prior to carrying out software verification activities, softwareengineers should have a clear understanding of the software product to be verified; they can applyeither black-box or white-box testing techniques as they yield similar defect detection rates.