A CASE STUDY ON AUTOMATED SOURCE-CODE-BASED TESTING METHODS

摘要

We present results of a case study on a testgeneration approach called Flow-optimized AutomatedSource-code-based unit Testing (FAST) which generates teststimuli from information available in the source code, inparticular taken from the detailed software interfaces. Thisallows automation of a significant part of testing, rangingfrom the test stimuli generation to the generation of the testreport. A huge number of stimuli can be generated exploringthe behaviour of the software under test under nominal andnon-nominal conditions. Symptoms like timeouts, unexpectedtermination, run-time exceptions, out-of-range conditions andmissing coverage are applied for defect detection. The goal ofthis study was to evaluate the FAST process in context of areal spacecraft flight software application and to get afeedback on its scalability regarding larger applications, itssensitivity on detecting defects in the code, the achievable testcoverage, its compliance with software standards andpotential limitations. We also consider the impact of codingstyle on suitability for automated testing. The results confirmthat the approach (1) provides acceptable code coverageresults without requiring manual intervention for testpreparation and execution, (2) raises the probability ofactivation of exotic fault conditions, (3) may provide hints onlocations in the code where robustness needs to be verified,and (4) identifies defects not found before by static analysisand intensive testing.