Coverage metrics for requirements-based testing.

摘要

In software development, validation that the software meets the customer requirements is accomplished through manual inspections and testing. Current practices in software validation rely on the engineering judgment of domain experts to determine whether or not the tests developed for validation adequately exercise the requirements. There is no objective way of determining the adequacy of validation tests. The work in this dissertation tackles this problem by defining objective metrics termed, requirements coverage metrics, that helps determine whether the behaviors specified by the requirements have been adequately tested during software validation.;We define coverage metrics directly on the structure of high-level software requirements. These metrics provide objective, implementation-independent measures of how well a validation test suite exercises a set of requirements. We focus on structural coverage criteria on requirements formalized as Linear Temporal Logic (LTL) properties. These criteria can also be used to automatically generate requirements-based test suites (test suites derived directly from requirements) so that the high-cost of manually developing test cases from requirements is reduced. To achieve this, we developed a framework that automates the generation of requirements-based test cases providing requirements coverage. Unlike model or code-derived test cases, these tests are immediately traceable to high-level requirements. We illustrate the usefulness of the proposed metrics and test case generation technique with empirical investigations on realistic examples from the civil avionics domain.;Another potential application of requirements coverage metrics---in the model-based software development domain---is to measure adequacy of conformance test suites. Conformance test suites are test suites developed to test the adherence of an implementation to its specification. We found that the effectiveness of existing adequacy metrics can be improved when they are combined with requirements coverage metrics. Test suites providing requirements coverage are capable of revealing faults different from test suites providing coverage defined by existing metrics. We support this claim with empirical evidence and statistical analysis illustrating the usefulness of requirements coverage metrics for measuring adequacy of conformance test suites.;To summarize, this dissertation introduces the notion of requirements coverage and defines potential metrics that can be used to assess requirements coverage. We make the following claims, supported by empirical evidence, regarding the usefulness of requirements coverage metrics: (1) Provides an objective measure of adequacy for software validation testing; (2) Allows for autogeneration of tests immediately traceable to requirements; (3) Improves effectiveness of existing measures of adequacy for conformance testing.