Reliability and validity of multiple choice examinations as a function of the number of options per item and student ability.

摘要

This study attempts to determine the extent to which test reliability and validity of multiple-choice examinations could be maximized by varying the number of options per item and the ability of the student. Three forms of the Vocabulary subtest from the Washington Pre-College Test Battery were given to 432 eleventh grade high school students in the Portland, Oregon metropolitan area. Each form consisted of 3-, 4-, or 5-option items. The least discriminating options were deleted from the original test to construct the other forms. Students were grouped into high, average, and low ability groups based on their estimated grade-point averages. KR-20s were calculated for each test form and ability group. The Tversky condition (Tversky, 1964), which states that testing time is proportional to test length was implemented by using the Spearman-Brown formula to determine the reliability for a lengthened version of the 3- and 4-option test forms given the total number of options found in the 5-option form.;KR-20s were statistically compared using the M statistic (Hakstian & Whalen, 1976). The validity coefficients were compared by using the U statistic (Marascuilo, 1966). The results of the study showed that significant differences (p ;Nonsignificant differences were found between reliability coefficients for high or average ability students. The optimum number of options across ability groups was three. None of the validity coefficients followed the trend expected. Possible explanation of the findings included: (1) The process of eliminating options to construct the different test forms did not affect the reliability; and (2) the restricted range of achievement of the subjects attenuated the magnitude of the reliability coefficients.