Cryptographic Verification of Test Coverage Claims

摘要

The market for software components is growing, driven on the 'demand side' by the need for rapid deployment of highly functional products and on the 'supply side' by distributed object standards. As components and component vendors proliferate there is naturally a growing concern about quality and the effectiveness of testing processes. White-box testing particularly the use of coverage criteria is a widely used method for measuring the 'thoroughness' of testing efforts. High levels of test coverage are used as indicators of good quality control procedures. Software vendors who can demonstrate high levels of test coverage have a credible claim to high quality. However verifying such claims involves knowledge of the source code test cases build procedures etc. In applications where reliability and quality are critical it would be desirable to verify test coverage claims without forcing vendors to give tip variable technical secrets. in this paper we explore cryptographic techniques that can be used to verify such claims. Our techniques have certain limitations which we discuss in this paper. However vendors who have done the hard work of developing high levels of test coverage can used these techniques (for a modest additional cost) to provide credible evidence of high coverage while simultaneously reducing disclosure of intellectual property.