Automating the Knowledge Assessment Workflow for Large Student Groups: A Development Experience

摘要

The paper presents our long-term experience at the University of Belgrade-School of Electrical Engineering in the development and use of the software system called Test, designed to automate the knowledge assessment workflow for large student groups in situations that usually require the use of pencil-and-paper testing. The test assembly is done interactively and/or automatically from a database of problems (questions, coding assignments, etc.) and previous tests. In order to enable objective and uniform knowledge tests, a rich set of parameterized problem selection criteria is made available to teachers. Preparation, scanning, and analysis of bubble sheets are highly automated and designed to work with plain paper and regular scanners. The Test system integrates with relevant learning management systems and the school's information system. Quantitative measurements have indicated improvements in both test quality of individual tests and uniformity of test quality across multiple tests, compared to the tests assembled manually according to intuitively applied criteria. The system is used for the introductory programming courses at the School. It has been developed through a number of diploma theses. Overall experience showed that the designed system provides an appropriate automation tool, a useful educational platform, and a valuable base for further research.