In a software life cycle, testing is crucial as it directly impacts the quality of the product. As the complexity and size of software products increase, testing cost also increases resulting in a growing need for efficient software testing. The challenge of striking a balance between the limited test resources and the desired product quality has become common in most modern software organizations. The objective of this paper is to present an approach that is geared towards higher product quality and lower testing cost. The overall strategy is to predict the fault proneness of the constituents of a software system and, direct the test resource requirements based on the fault proneness. In the presented approach, a large software system is decomposed into smaller units that can be tested independently. This specific type of code unit is referred to as a component; it can be an executable file or any module which is part of the overall software product. For each component, a large number of software metrics such as code complexity, code criticality, code churn, historic defect metrics etc are considered as contributors to risk. Statistical models, such as neural networks and logistic regression, are then used to predict future risk by correlating the combination of these metrics to a measure of failures fixed. Relationship amongst components is also considered to determine the impact of the risk that one unit has to the other. Components are calibrated on the basis of their predicted risk of having failures. A model based on constraint satisfaction problems has been developed to minimize risk and maximize test efficiency for a given set of resource and risk constraints. The paper will conclude with a case study conducted at Microsoft in the Windows Serviceability group. Key factors leading to software failures were investigated and the presented approach has been applied for achieving test efficiency. The paper will include an elaboration on the end-to-end process of achieving test efficiency, risk assessment, quantifying relationship between components and, finally the method of coming up with test recommendations. Results are presented where reduction in test efforts is achieved with minimal risk to product quality.
展开▼
