Effects of implementing performance assessments on student learning: Meta-analysis using HLM.

摘要

Multiple-choice tests had dominated in education for several decades. In reply to a number of negative consequences of traditional multiple-choice tests, researchers proposed measurement-driven-reform with performance assessments. The main rationale for this movement is that changing assessment from multiple-choice tests to performance assessments will change instructional practices in the classroom, that encourage higher-order thinking skills, and as a result, they will improve student learning. In this study, the consequences of implementing performance assessments were investigated by conducting a meta-analysis using HLM (Hierarchical Linear Modeling). Specifically, whether or not performance assessments had a positive impact on student learning was examined.; The results of HLM analyses revealed that performance assessments improved student learning on average. Moreover, the variances across the studies were dramatically reduced after the removal of sampling errors, correction of unreliabilities, and controlling the 12 local variables. For the reliability uncorrected data, the total variances was reduced by 57.3%, and for the reliability corrected data, 70.7% of the total variances was reduced. However, the results of Chi-square tests indicated that there still exist variabilities across the studies, indicating that the impacts of performance assessments on student learning cannot be generalized. In other words, the effects of performance assessments on student learning are locally valid.; Among the 12 local variables, the number of months after implementing performance assessments, cognitive learning outcome, average-performing students, and use of performance assessments as an instructional tool were statistically significant predicting effects of performance assessments on student learning. In addition to these four variables, American and one group pretest-posttest design without random assignment were statistically significant for the uncorrected data, and the variable of high-performing students was significant for the corrected data.; Furthermore, this study found out the best (worst) combination of the local variables producing the largest (smallest) effect. The combination of longer implementation of performance assessments (number of months after implementing performance assessments), cognitive learning outcome (types of learning outcome), average-performing students (levels of student ability), Korean (nationality), one group pretest-posttest design without random assignment (types of research design), and use of performance assessments as an instructional tool produces the largest effect size. In contrast, the combination of shorter implementation of performance assessments, psychomotor learning outcome, a mixed level of students, American, posttest-only control-group design with random assignment, and no use of performance assessments as an instructional tool produces the smallest effect size.