Crash Scene Investigation: Instrumentation and Postmortem Analysis for Deployed Applications

摘要

Debugging is a difficult, error-prone, and tedious task. Debugging, testing, and verification can account for 50-75% of a software project's cost; these costs can grow even higher in some cases. Nevertheless, or perhaps partly because of this, complex production software inevitably contains bugs, and post-deployment failures are common. These production failures increase the cost of bug repair substantially, as they are difficult to reproduce and are initially observed by non-experts. Detailed postmortem crash reports can theoretically alleviate this problem. In particular, a developer would greatly benefit from a concrete, reproducible, full execution trace. Sadly, full execution tracing is usually impractical for complex programs. Even for simple code, the overhead of full tracing may only be acceptable during in-house testing.;The first part of this dissertation describes new techniques and tools for lightweight instrumentation of deployed applications. We describe techniques that trace program paths and that gather program coverage data at various granularities. We also present new algorithms for optimizing program coverage instrumentation based on customized requirements. Across all of these approaches, our techniques focus on extremely low-overhead control-flow tracing, and take advantage of readily-available information from failure reports (e.g., stack traces) to avoid redundant tracing and prioritize information to maintain when trace data must be limited due to tight overhead requirements.;The second part of this dissertation defines analysis techniques that operate on post-deployment failure reports. These reports may contain a variety of elements, but usually will not contain a full trace of run-time information from the failing execution. Thus, our analysis techniques are specifically designed to grapple with the imperfect information contained in production-run failure reports (including, but not limited to, our own tracing techniques). Our techniques aid developers in the difficult task of debugging post-deployment failures by restricting the possible program code a developer needs to consider, and by answering user control-flow questions about the failing execution.