A Case Study on Automated Source-Code-Based Testing Methods

摘要

We present results of a case study on an test generation approach called Flow-optimized Automated Source-code-based unit Testing (FAST) which generates test stimuli from information available in the source code, in particular the detailed software interfaces.. This allows automation of a significant part of testing, ranging from the test stimuli generation to the generation of the test report. A huge number of stimuli can be generated exploring the behaviour of the software under test under nominal and nonnominal conditions. Symptoms like timeouts, unexpected termination, run-time exceptions and missing coverage are applied for defect detection. The goal of this study was to evaluate the FAST process in context of a real spacecraft flight software application and to get a feedback on its scalability regarding larger applications, its sensitivity on detecting defects in the code, the achievable test coverage, its compliance with software standards and potential limitations. We also consider the impact of coding style on suitability for automated testing. The results confirm that the approach (1) provides acceptable code coverage results without requiring manual intervention for test preparation and execution, (2) raises the probability of activation of exotic fault conditions, and (3) may provide hints on locations in the code where robustness needs to be verified.