Evaluating the Learning Effectiveness of an Online Information Literacy Class Based on the Kirkpatrick Framework

摘要

The purpose of this study was to use the Kirkpatrick four-stage framework to evaluate the learning effectiveness of the online general education information literacy materials a€?Library and Information Utilization,a€? initiated by the Ministry of Education (MoE) for higher education in Taiwan, under the Universitya€?s General Education e-Learning Materials Promotion and Instructional Application Plan 2011-2013. The study population consisted of 206 students at Tatung University and the course evaluation spanned two academic years. It produced a total of 194 online questionnaires used to evaluate levels 1-4 in the Kirkpatrick model. The study used a mixedmethod approach, including a mainly quantitative online questionnaire, followed by ten semi-structured interviews of students who took the course between the 2010 fall semester and the 2012 fall semester to evaluate level 3 and level 4. Empirical data collected from the course tasks were also used to evaluate level 2. In general, the results demonstrated that students were more confident in their future study and daily life after learning the online material. Based on the Kirkpatrick model, the results provide favourable evidence for course effectiveness and its value. The results also demonstrated a reasonable Return of Expectation (ROE) and Return on Investment (ROI) to the MoE and to the University in terms of cost evaluation versus potential benefits. The findings will be beneficial to the MoE in the future development of digital learning materials for higher education, and can also be useful to instructors in developing interventions to improve the design and delivery of online courses. This study suggests that the Kirkpatrick model is a workable instrument to measure the effectiveness of an IL course and to evaluate if the information competencies have been maintained and made useful for further study and life-long learning.