Mutant Accuracy Testing for Assessing the Implementation of Numerical Algorithms

摘要

Despite their widespread use, implementations of numerical computing algorithms are generally tested manually with fuzzily defined thresholds determining success or failure. Modern software testing methods, such as automated regression testing, are difficult to apply because both test oracles and algorithm output are approximate. Based on the observation that high accuracy numerical algorithms appear to be fragile by design to errors in their parameters, we propose to compare the error of target implementations to mutated versions of themselves with the expectation that the mutants will suffer degraded accuracy. We test the idea on Matlab implementations of some basic numerical algorithms, and find that most mutants are worse while the few which are better show a distinctive pattern of mutation.