Comparison of Student Evaluation of Teaching Results when Stratified by Protocol, Course Content, and Course Structure

摘要

Focusing on the mechanical engineering undergraduate program at the University of Alberta, this study attempts to quantify biases in student evaluation of teaching (SET) results that could be attributed to SET protocol, course content, and course delivery mode. SET results were compiled for five academic years of paper based SET evaluation and one semester of online SET evaluation. 20 core undergraduate courses were included; class size from 70-130; 35 professors. Statistical analysis included compilation of frequency histograms, determination of means and standard deviations, and rank-sum tests for significant differences based on aggregated data for several stratifications. Results showed significantly reduced response rate for online SET when compared to paper; ratings of professor evaluation were not different. No significant differences were found when results were compared on the basis of course content or delivery mode. Our aggregated data showed SET protocol lead to lower response rate, but not significant differences in instructor evaluation. Course content and delivery mode did not manifest in significant changes in SET results. Typical variability in instructor rating was 0.4/5.0 considering all data. Administrators and senior faculty should be aware of these results when ascertaining instructor performance. Although focused on one department, the study is a first step in a larger evaluation of SET in engineering. The study identified key variables that must be further evaluated.