综合应用现代测量理论,分析中国初中学业水平考试中固定分数法标准设定的信度.分别从中国东中西部地区3个区县随机抽取3 000名初三学生为被试,对被试的数学学业水平考试数据进行分析.采用的信度指标包括经典测量理论的决策一致性系数( Kappa )、概化理论的等级线决策信度 Φ ( λ )和项目反应理论的信息量 I ( θ ).研究结果显示,固定分数法标准设定下,决策信度接近0.7;等级线决策信度大于0.7,大部分在0.8左右;分界点的信息量大部分低于16.这些结果表明,中考采用固定分数法进行标准设定的质量一般,对于学业水平考试这种高利害性考试来说需要进一步提高.%The main purposes of this study were to examine the reliability of current standard setting method of Fixed Score in academic level examination for Secondary School. Using stratified random sampling design to select three counties from the East, Middle, and West of China respectively, 3 000 students of each county are chosen. The data from the Academic Level Examination for Secondary School was used. A comprehensive application of modern measurement theory to analyze the reliability indicators of standard setting, including Decision Consistency Index (Kappa) in Classical Testing Theory (CTT), Cut-score Dependability Φ(λ)in Generalizability Theory (GT), and the amount of information index I(θ)from Item Response Theory. The results showed that: (i) the Decision Consistency Index of current Entrance Examination for Secondary School approaching 0.7; (ii) Φ(λ)of the cut-scores was greater than 0.7, mostly beyond 0.8; (iii) I(θ)were less than 16 regardless of methods to Fix Score or Fix Ratio. All these results suggested that the quality of existing tests' standards setting method of Fixed Score was barely satisfactory, and it should be improved for high-stakes examinations.
展开▼
