Measuring Instructional Differentiation in a Large-Scale Experiment

摘要

This study operationalizes four measures of instructional differentiation: one for Grade 2 English language arts (ELA), one for Grade 2 mathematics, one for Grade 5 ELA, and one for Grade 5 mathematics. Our study evaluates their measurement properties of each measure in a large field experiment: the Indiana Diagnostic Assessment Tools Study, which included two consecutive cluster randomized trials (CRTs) of the effects of interim assessments on student achievement. Each log was designed to measure instructional practices as they were implemented for eight randomly selected students in the participating teachers' classrooms. A total of 592 teachers from 127 schools took part in this study. Logs were administered 16 times in each experiment. Item responses to the logs were scaled using the Rasch model and reliability estimates for the differentiation measures were evaluated at the log level (observations within teachers), the teacher level, and the school level. Estimated reliability was above.70 for each of the log- and teacher-level measures. At the school level, reliability estimates were lower for Grade 5 ELA and mathematics. The variance between teachers and schools on the scaled differentiation measures was substantially less than within-teacher variation. These results provide preliminary evidence that teacher instructional logs may provide useful measures of instructional differentiation in elementary grades at multiple levels of aggregation.