CORRELATING PROBLEM/PROCESS EXAM QUESTION COMPLEXITY TO ANTICIPATED EFFORT: A MODELING PROTOCOL

摘要

This paper presents the initial investigation of the use of complexity as a surrogate for problem difficulty in predicting the effort or point value of an exam problem. In previous research, complexity of graph-based models has been used to predict market value of products using function models and to predict assembly time from connectivity graphs. This research investigates the potential of applying graphical representations and complexity metrics for exam problem solutions using expert assigned values as an appropriate method to offer point values for new exam questions. The factors and sources of problem difficulty are examined and compared to the structural complexity of a graphical representation of the problem solution. Specifically, this paper presents a protocol for developing the graphical representation. Multiple participants used the protocol to create graphical models of three exam questions to test and validate the usability of the protocol. A secondary protocol was tested to improve the rater agreement for use of the protocol. This protocol will be used for transforming exam problems into graphical models that can be analyzed with the connectivity complexity metrics. These metrics will be used to create predictive models for point assignments based on historical data.