IMPROVING TEST AUTOMATION BY DETERMINISTIC METHODS IN STATISTICAL TESTING

摘要

Statistical testing is of increasing interest because it allows full test automation - from test generation to evaluation - and hence reduces significantly the human test effort, while allowing a much broader test range. However, automatically generated tests based on a statistical approach have to cope with the "oracle problem" and the "small target problem". The oracle problem represents the fact that software cannot conclude on the correctness of the derived results by itself, while the small target problem consists in the challenge of hitting sporadic, but important test conditions in a large input domain. In the worst case this problem can result in zero-probability for specific test conditions. Although deterministic testing methods in principle do not suffer from the small target problem, they suffer from an oracle problem as well, which is the problem to know which test cases are needed. In general, the test criteria need to represent all viable questions a test engineer wants to pose on the respective system under test. Therefore deterministic methods are best at pointing out anticipated faults (as far as "anticipated" by a test engineer) while statistical methods can also reveal non-anticipated faults.