Effects of test linking methods on proficiency classification: UIRT versus MIRT linking.

摘要

The major purpose of this study was to show educational policy implications of psychometric decisions on educational measurement by exploring the effects of selecting different linking approaches---UIRT vs. MIRT linking---on proficiency rate changes. The result shows that different linking approaches and different choices of proficiency classification models produce different conclusions on the educational progress inferred from increased or decreased proficiency rates across years.;In this study, a fixed common item parameter (FCIP) linking method was selected to link two years' test data in both the UIRT and MIRT linking approaches. Five random sample data (RSD) sets of 10,000 samples were selected for two years and each RSD set of one year (2005) was linked to its matched RSD set of 2006. For an UIRT approach, the PARSCALE program was employed. The two years' test data were calibrated separately by running PARSCALE and 2005 results were recalibrated by fixing the common items with the item parameters calibrated in 2006 data. For MIRT linking, each RSD set was calibrated by running the BMIRT Program. For FCIP MIRT linking, the BMIRTanchor program was employed.;This study found that different linking approaches and different decision approaches to proficiency classification produced different results. Overall, the UIRT approach was favorable to the 2006 students. While there was little change in proficiency rate between 2005 and 2006 using the UIRT approach, both the MIRT compensatory and conjunctive approaches resulted in a decreased proficiency rate in 2006 compared to 2005 for the 20th percentile classification criterion and no statistically significant difference between two years for the 50th percentile classification criterion. This result strongly suggests the importance of selecting a linking method and a proficiency classification approach when evaluating educational progress by the change of proficiency rate.