CODE COVERAGE ISN'T QUALITY, IT ISN'T EVEN COVERAGE

摘要

If anybody ever says "I know our code has high quality because we achieved 100% code coverage," smack them. There really is no relationship between code coverage and quality because code coverage does a dreadful job telling you whether or not tests have been thorough. Good test coverage only comes from doing as many interesting things as possible that are good at exposing flaws in code. Code coverage reports, when used as a metric of quality, hide the useful tests. This paper shows real examples of code coverage reports that showed 100% coverage that were completely, and entirely useless. Alternate tests will be presented that demonstrate how sometimes directing one's attention in the opposite direction implied by code coverage actually yields better test generation and more confidence in what the test suite addresses.