Collaborative E-Test Construction: Using Predicted Response-Time and Score Distributions to Improve Reliability

摘要

Analysis of collaborative e-test construction identified the number of test-authors as the most important factor in test validity, while test reliability depends more on participation of an expert. Based on these findings, a collaborative e-test construction system was developed that uses predicted response-time and score distributions to improve the reliability of tests constructed by novice test-authors. A gamma distribution is used as the predicted response-time distribution, and a mixed model of binomial distributions is used as the predicted score distribution. An experiment in which a novice and an expert test-author each constructed tests by using and not using these predicted distributions showed that those constructed using them were more reliable, although those constructed by the expert had even higher reliability.