A Case Study on Automated Source-Code-Based Testing Methods

摘要

We present results of a case study on an testgeneration approach called Flow-optimized AutomatedSource-code-based unit Testing (FAST) which generates teststimuli from information available in the source code, inparticular the detailed software interfaces.. This allowsautomation of a significant part of testing, ranging from thetest stimuli generation to the generation of the test report. Ahuge number of stimuli can be generated exploring thebehaviour of the software under test under nominal and nonnominalconditions. Symptoms like timeouts, unexpectedtermination, run-time exceptions and missing coverage areapplied for defect detection. The goal of this study was toevaluate the FAST process in context of a real spacecraft flightsoftware application and to get a feedback on its scalabilityregarding larger applications, its sensitivity on detectingdefects in the code, the achievable test coverage, its compliancewith software standards and potential limitations. We alsoconsider the impact of coding style on suitability for automatedtesting. The results confirm that the approach (1) providesacceptable code coverage results without requiring manualintervention for test preparation and execution, (2) raises theprobability of activation of exotic fault conditions, and (3) mayprovide hints on locations in the code where robustness needsto be verified.