Web site usability: A case study of student perceptions of educational web sites.

摘要

The purpose of this research study was to understand the construct of usability from the perspective of 74 students enrolled in six online courses offered by one online and distance learning program at a large, public university in the Midwest. Six courses, designed and developed by two different groups, professional and nonprofessional developers, were selected. The study used both quantitative and qualitative measures to record the experiences of students enrolled in the six online courses. First, the courses were evaluated using Nielsen's (1994, 2000, 2002) heuristics as operationalized by the Xerox Heuristic Evaluation Checklist (1995) as a standard measure of usability, then rank-ordered by heuristic evaluation score. Eachus and Cassidy's (2006) Computer Use Self-efficacy Scale was used as a pre-course survey to measure students' computer self-efficacy prior to beginning their online course. Stewart, Hong, and Strudler's (2004) Quality of Web-based Instruction was used as a post-course survey to measure student satisfaction with their online course experience. A subset of 29 students participated in usability testing sessions in the usability lab. A think-aloud protocol provided qualitative data in the form of verbal reports, eye-tracking recordings provided data confirming the think-aloud protocol data, and a time-error log provided "time to complete tasks," and "error rate" data as students completed seven typical tasks required to successfully participate in an online course. A summary, debriefing interview with each student was conducted to record any additional student comments and any student recommendations for improving the courses. Qualitative data were examined for themes and a coding scheme was created. This coding scheme, which illustrated the issues specific to educational web sites, was compared to Nielsen's (1994, 2000, 2002) heuristics to evaluate whether Nielsen's (1994, 2000, 2002) heuristics, widely accepted as the standard for the design and development of business and commercial websites, also apply to educational web sites. Design and development guidelines for educational web sites were written by the researcher based on the study findings. These guidelines were mapped to Nielsen's heuristics as operationalized by the Xerox Heuristic Evaluation Checklist (1995).;The results of the quantitative and qualitative measures used were analyzed by course and course development type. The most significant results of this study came from the analysis of the variables according to course development type. The results of the study findings include that the course design type, professional or nonprofessional, was related to usability as measured by students' error rates, Nielsen's heuristic evaluation scores, and student satisfaction scores. The professionally-developed courses were found to be significantly higher in usability than the non-professionally-developed courses by task error rate, Nielsen's heuristic evaluation score and student satisfaction scores. The analysis of students' verbal reports resulted in three times as many positive comments for the professionally-developed courses when compared to the positive comments for the nonprofessionally-developed courses.;The results of the quantitative and qualitative measures used were also analyzed by course. When comparisons were made between courses using courses as the unit of analysis the findings were different. The rank-order of courses was mixed between course types when compared by error rates. The Nielsen's heuristic evaluation scores as measures of usability for educational web sites were not consistent with students' judgments of course usability as measured by error rate scores. There was no relationship between the usability ranking of courses by Nielsen's heuristics and usability as judged by students' error rates. However, an analysis of students' verbal reports identified 52 common themes and confirmed the importance of Nielsen's heuristics in educational course design.;The correlation between the self-efficacy score and error rate means was nonsignificant. The correlation between self-efficacy and error rate was small; very close to zero. There was a small positive correlation between student satisfaction and usability as measured by error rates.;Based on the analysis of the study variables according to course development type, the results of this study found that Nielsen's usability heuristics, a respected evaluation tool used primarily to measure the usability of commercial web sites, can be used to evaluate instructional web sites and used to differentiate between levels of usability in the same way usability is judged by students.