Measuring the Effect of Blended Learning: Evidence from a Selective Liberal Arts College

摘要

The combination of online homework, video-based lecture material, and in-class problem solving that we used resulted in larger increases in TUCE scores for the blended courses than for the controls. This stands in contrast to the few studies of blended courses within economics, so we consider some potential explanations here. One possibility is that we maintained the same amount of class time in blended and nonblended courses, so students do not give up face-to-face interaction. The treatment may have led to larger increases in TUCE scores simply because the pedagogy required students to spend more time studying. Another is that our participants are drawn from an unusual student population, averaging within the top 10 percent of high schoolers on standardized tests and starting the course with a fair amount of economic intuition, as measured by the TUCE. Another possibility is that we were biased as researchers and "taught to the test." This could be a concern since we were unable to conduct our study with the same instructors teaching both traditional and blended courses, but the anonymity of the study helped ensure that we could not spend extra time with students who scored low on the pretest, and a shared set of coverage goals across the department that need to be met within the ten-week term would make it difficult to cover TUCE questions that are not part of the usual curriculum. Other studies (Brown and Liedholm 2002; Emerson and Taylor 2004; and Dickie 2006 on the use of experiments) find evidence that certain student subgroups, particularly higher-achieving students, benefit more from nontraditional techniques. Preliminary analysis does not show interaction effects between our treatment and high standardized test scores, but given the selective nature of our institution, we may not have the variation necessary to satisfactorily test the hypothesis. Future researchers may benefit from the finding that students' self-reported effort on the TUCE (as measured by SOS scores) does not vary between the pretest and the posttest, and students attribute similar importance to both tests despite the posttest being administered soon before a final. One might worry that part of the typical improvement in TUCE scores is simply that students care more about the test once they have actually taken the course, but we find that this is not the case. Although a student putting more value in the TUCE is linked to higher TUCE scores in one of our regression specifications, this effect is very small, and would have little impact anyway since students on the whole do not place more weight on the posttest. In future work, we plan to explore the question of potential interaction effects more, and whether a grade incentive would affect students' answers on the TUCE, using a within-subjects methodology based on a subset of TUCE questions that we included on final exams.