Reflective item types in a high-stakes assessment of English for Academic Purposes: An examination of the method effect.

摘要

Several studies (Bachman, 1990; Fox, 2003; Kenyan, 1995; Upshur and Turner, 1999) have examined the systematic effect of item types on scores. However, although writing researchers (Elbow, 1991; Emerson, 1983; Faigley, 1994; Fulwiler, 1988; Moon, 1999) have generally agreed that reflection affects writing, no research to date has examined the effect of reflective item types on the discourse produced in a testing context. This study used a mixed method approach (Creswell, 1994; Greene, et. al, 1989) to examine the effect of reflective test items on the writing scores and test taker composing strategies in the context of a topic based, fully-integrated EAP test. Phase 1 examined scores using aggregate data from 116 test takers. Phase 2 used grounded theory (Glaser and Strauss, 1967) to investigate ten test takers' composing strategies. Although the results must be interpreted cautiously, evidence suggested that adding reflective item types may have (1) clarified the task, resulting in increased scores for some test takers and (2) had a positive effect on composing strategies.