Threats to Validity in a Study of the Effects of Hypermedia Instruction on Learning Outcomes - a Switched Replications Experiment

摘要

Educational researchers in university settings face many difficulties in trying to conduct controlled action research studies on the effects of hypermedia on learning outcomes, where threats to validity and reliability are often beyond the influence of the investigator. Switched Replications experiment and another, semester-long study, where all students received hypermedia instruction in which it was embedded, were specifically conducted to address several validity threats. In each of the two weeks of the experiment, one group of students received a hypermedia lecture, while the other group received a conventional lecture on the same topic. The two lectures were immediately followed by a test. Two instructors were involved. The hypermedia-instructed group performed significantly better both times, regardless of which instructor made presentations. However, there were no statistically significant differences in overall academic achievement of the students in the course, offered in a hypermedia mode for 12 out of its 13 weeks. This indicated that the choice of instructional media had a strong effect on student achievement, while the effect of instructor differences was negligible. Selection bias, novelty factor, differences in instructional design and social threats to the internal validity of the study were also rejected as a possible explanation for the observed differences in achievement.