Investigating whether a value-added teaching effectiveness model designed for traditional classrooms can be used to measure online teaching quality

摘要

The study reported here focused on assessing teacher quality in online environments. The purpose of the study was to explore the feasibility of using the same method Tennessee currently uses to gauge teaching quality of traditionally-delivered courses to determine teaching quality in the online environment. Research questions were: 1) Is there a significant difference in program effects of traditional classrooms (as measured by end-of-course scores for a sample of traditionally-taught students in a Tennessee school district) and online classrooms (as measured by end-of-course scores for a sample of Tennesseeu27s online students)? 2) Do program effects between traditional and online environments vary significantly by subject area (i.e., Algebra I, Biology, and English 1)? 3) Do Tennessee educators perceive that Tennesseeu27s model for teacher-effect scores can be used equally well in both traditional and online environments? 4) What factors and strategies do educators perceive should be considered in determining teaching quality in the traditional and online environments? Research questions 1 and 2 were addressed by comparing EOC scores from students in each program. EOC scores from 162 students in a Tennessee online program were compared with a sample of 162 students from a Tennessee school district that were systematically selected to match the online sample in several important characteristics (e.g., socio-economic levels, indicators of prior achievement). A regression analysis was used to identify variables that contributed significantly to studentsu27 EOC scores, and effects of the two programs were compared by using an Analysis of Covariance (ANCOVA) to control for contributions of these variables. With these variables controlled, no significant differences existed between online and traditional programs in any content areas, except in Algebra I when only NCE scores were considered as a covariate. To address research questions 3 and 4, 68 Tennessee educators completed a survey with open-ended and Likert-scale items. Survey data indicated a lack of understanding for Tennessee’s teacher-effect model and a general perception that traditional teacher quality indicators cannot be used to assess teachers in the online environment. Implications of these findings and directions for future research are discussed.