An empirical sensitivity analysis of value-added teachers' effect estimates to hierarchical linear model parameterizations.

摘要

There is currently a considerable amount of debate among proponents of value-added modeling regarding the most appropriate models for estimating teachers' effects on student academic gains. Value-added models differ in their complexity and stringency of data requirements. There are many competing methods for computing estimates of teachers' effects, and none are universally accepted as optimal at this time. Even within a specific class of models, such as hierarchical linear models, there are many different options regarding the exact parameterization of the models to be used. This methodological study investigated the sensitivity of estimated teachers' effects to different hierarchical linear model parameterizations to ascertain whether increased model sophistication is likely to lead to substantively different estimated teacher effects. The impact of different model specifications on (a) the rank ordering of teachers based on estimated teachers' effects, (b) the identification of those teachers who are farthest below or above expectation based on teacher effect estimation, (c) the proportion of variance in student achievement change attributable to teachers' effects, and (d) indices of model fit were explored using large scale real achievement data.; The primary findings of this dissertation are that (a) the seven different model types compared differ very little from one another in their estimation of teachers' effects with the exception of the adjusted cumulative effect model, (b) the percent of variability in student academic gains attributed to teachers' effects is much larger as estimated with the cumulative effects models as opposed to other models included in this study, and (c) the predictor variables used in this study appear to improve model fit and have a detectable, but marginal, impact on estimating relative teacher performance.