Call-Stack Coverage for GUI Test Suite Reduction

摘要

Graphical user interfaces (GUIs) are used as front-ends to most of today''s software applications. The event-driven nature of GUIs presents new challenges for testing. One important challenge is test suite reduction. Conventional reduction techniques/tools based on static analysis are not easily applicable due to the increased use of multi-language GUI implementations, callbacks for event handlers, virtual function calls, reflection, and multi-threading. Moreover, many existing techniques ignore code in libraries, and fail to consider the context in which event handlers execute. Consequently, they yield GUI test suites with seriously impaired fault-detection ability. This paper presents a reduction technique based on the call-stack coverage criterion. Call stacks may be collected for any executing program with very little overhead. Empirical studies in this paper compare reduction based on call-stack coverage to reduction based on line, method, and event coverage, including variations that control for size and optional consideration of library methods. These studies show that call-stack-based reduction provides an excellent tradeoff between reduction in test suite size and loss of fault detection effectiveness. Additionally, an analysis of the relationship between coverage elements and fault-revealing test cases is presented.