Open-Book Testing: a Method to Teach, Guide, and Evaluate Testers

摘要

One technique for teaching and evaluating students in an academic setting is to give them an exam. One method is a "closed-book" exam, where students are tested on what they know and are not allowed to consult resources. The instructor lists questions they want the student to answer, and either the student answers them correctly or not. If a student brings a resource and consults it during the exam, it is often grounds for dismissal and expulsion. In an "open-book" exam, however, students can bring and consult references. Unlike closed-book exams, open-book exams do not just test knowledge, but also the student's ability to acquire the knowledge on demand and apply it to the question. As curriculum content was reviewed at Singapore's Nanyang Technological University in 1998, a survey of students showed that in order to develop creative and independent thinkers, "more open-ended tasks that reflect real-life situations - questions involving problem-solving should be incorporated in examinations, especially for higher level education." (Han C, 1998). Another study said that open-book "removes much of the fear and emotional block encountered by students during examination, while, at the same time, it emphasizes practical problems and reasoning rather than recall of facts." (Theophilides & Dionysiou - 1996). In my ten years as a tester and test manager, I can attest to the same experience as I experiment with open-book techniques in my role as Managing Test Lead for a software testing lab specializing in rapid and heuristic exploratory test techniques This paper is about how instructors and test managers can use an Open-Book Testing paradigm on software projects to: 1. Teach: Acquaint testers with the main activity of testing -asking questions of the software; 2. Guide: Give testers a context or a mission to frame their questions and get them familiar with the product quickly; 3. Evaluate: Assess testers' questioning and critical thinking skills.