Combinatorial testing, random testing, and adaptive random testing for detecting interaction triggered failures

摘要

Context: Software behavior depends on many factors, and some failures occur only when certain factors interact. This is known as an interaction triggered failure, and the corresponding selection of factor values can be modeled as a Minimal Failure-causing Schema (MFS). (An MFS involving m factors is an m-MFS.) Combinatorial Testing (CT) has been developed to exercise ("hit") all MFS with few tests. Adaptive Random Resting (ART) endeavors to make tests as different as possible, ensuring that testing of MFS is not unnecessarily repeated. Random Testing (RT) chooses tests at random without regard to the MFS already treated. Cl' might be expected to improve on RT for finding interaction triggered faults, and yet some studies report no significant difference. CT can also be expected to be better than ART, and yet other studies report that ART can be much better than RT. In light of these, the relative merits of CT, ART, and RT for finding interaction triggered faults are unclear.