The application of cluster filtering to operational testing of software.

摘要

A central activity in software testing is the problem of finding program faults. The observation-based testing methodology can be applied to this activity. Experimental results show that the use of cluster filtering to selectively identify test cases in the observation-based testing methodology is a reliable and efficient technique for the identification of failures in a population of program executions generated during late stage system testing.; A traditional method for identifying program faults is partition testing. Partition testing uses test cases derived from functional or structural subdomains. The method suffers from difficulty in forming clusters, the lack of correlation of the subdomains with faults, and the fact that the faults found may not be representative of the failures commonly seen by users.; Observation-based testing is an alternative methodology for software testing. It collects a population of runs from actual program usage. Any failures in this population will be representative of failures seen by the customer. The cluster filtering technique uses cluster analysis to partition the population based on the runs' execution profiles. It employs a sampling method to select test cases from that partitioning.; Observation-based testing has been shown to be a viable method for efficiently estimating program reliability. This work applies the method to debug testing. A series of experiments were undertaken to evaluate its utility. The results of those experiments show that (1) failed executions are concentrated in an identifiable clusters, namely those of small size; (2) the probability of finding any of the failed executions is, on the average, no worse than random sampling and in many cases is guaranteed; (3) the process is substantially more effective and efficient than random sampling at finding failed executions; and (4) the adaptive sampling method takes advantage of the homogeneity of the clusters to find more failed executions.