A G-Theory Analysis of Rater Effect in ESL Speaking Assessment

摘要

The current status of English as an international language has come with challenges to the native speaker norms and raised the relevance of localized varieties in language assessment. This preliminary study investigates whether native English-speaking (NS) and non-native English-speaking (NNS) raters differ in their effect on score reliability in ESL speaking assessment. A generalizability theory analysis indicated that, although NS and NNS raters exhibited similar severity patterns across all students, they interacted with the students in different ways. This article also discusses the implications for assessment practice and directions for future research.