Automatised Examination of Programming Courses - Design, Development and Evaluation of the AutoExam Online System

摘要

Traditional hand-written exams are still the main assessment method in programming courses at Swedish universities. Several conducted research studies indicates that computer based examination is a more natural environment for student examination and that the most authentic form of examination for programming courses should be to solve problems at a computer. The aim of this study is to describe the development of the web-based AutoExam system and discuss how the software system might contribute to modern programming education. AutoExam is designed and deployed as an online system and the artefact is constructed by using the Python programming language and the Django framework with a three folded Model-View-Controller division. As a main approach for this study methods defined in Design Science to be able to cover all the phases in the design, implementation, evaluation and communication of the AutoExam software artefact. The implemented automatised assessment system was evaluated in test examination by students with basic skills in the Python programming language. Semi-structured interviews were conducted with two programming teachers at the Department of Computer and Systems Sciences at the Stockholm University. Answers from the students' tests were analysed by using the One-Way Anova analysis method. The results were insignificant but showed a slightly better result for answers that were generated in the AutoExam environment. Both the interviewed teachers posit that there are positive aspects of computer generated examination answers both from the teachers' and the students' point of view. Our recommendation is that the kind of automatised examination of programming skills that is described in this article ought to be tested and evaluated in larger student groups and during longer time than what has been done in this limited study.