Automated Software Testing Based on Semantic Distance

摘要

Automatic student program assessment is found in both grading and tutoring systems; its purpose is to verify correctness in addition to programming bug localization and fixing. Respective assessment tools should therefore deal with logical programming errors in a way that is comparable to the functionality of modern compilers, which localize syntax errors by highlighting erroneous code in addition to delivering messages that help programmers understand and correct possible syntax errors. Because the number of possible program implementation variants, which are the correct solutions for a given problem, can be quite large, issues can arise when comprehensive assessments are performed. This prompts a novel approach to student program assessment. In this paper, we analyze approaches to student program assessment according to reliability of verification, bug localization and fixing as well as freedom of design. A novel method, described in this paper, localizes bugs precisely and provides minimal corrections, which are made according to students' intentions. The experimental evaluation of the method shows that it is reliable with respect to both false and missing alarms and that it also efficiently localizes and corrects programming bugs.