Pairwise Testing: A Best Practice That Isn't

摘要

Pairwise testing is a wildly popular approach to combinatorial testing problems. The number of articles and textbooks covering the topic continues to grow, as do the number of commercial and academic courses that teach the technique. Despite the technique's popularity and its reputation as a best practice, we find the technique to be over promoted and poorly understood. In this paper, we define pairwise testing and review many of the studies conducted using pairwise testing. Based on these studies and our experience with pairwise testing, we discuss weaknesses we perceive in pairwise testing. Knowledge of the weaknesses of the pairwise testing technique, or of any testing technique, is essential if we are to apply the technique wisely. We conclude by re-stating the story of pairwise testing and by warning testers against blindly accepting best practices.