Beyond plain video recording of GUI tests: Linking test case instructions with visual response documentation

摘要

Information systems with sophisticated graphical user interfaces are still difficult to test and debug. As a detailed and reproducible report of test case execution is essential, we advocate the documentation of test case execution on several levels. We present an approach to video-based documentation of automated GUI testing that is linked to the test execution procedure. Viewing currently executed test case instructions alongside actual onscreen responses of the application under test facilitates understanding of the failure. This approach is tailored to the challenges of automated GUI testing and debugging with respect to technical and usability aspects. Screen recording is optimized for speed and memory consumption while all relevant details are captured. Additional browsing capabilities for easier debugging are introduced. Our concepts are evaluated by a working implementation, a series of performance measurements during a technical experiment, and industrial experience from 370 real-world test cases carried out in a large software company.