Self adapted testing as formative assessment: Effects of feedback and scoring on engagement and performance.

摘要

This dissertation investigated the feasibility of self-adapted testing (SAT) as a formative assessment tool with the focus on learning. Under two different orientation goals---to excel on a test (performance goal) or to learn from the test (learning goal)---I examined the effect of different scoring rules provided as interactive feedback, on test takers (TTs) choice behavior, performance and engagement. Results indicated that choice behavior differed under the two orientation goals, with score maximization behavior observed under the performance goal, and more exploration behavior was observed under the learning goal. Engagement did not differ under the goal manipulation, which suggests that SAT with a learning goal can be as engaging as when there are external incentives to succeed.;The scoring rules that I applied in this study were characterized such that they differ in the weight they award to a correct answer (equal or unequal weight), crossed with a framing of the score as either monotonically increasing or not. An unweighted scoring rule, the number right rule (NR), is monotonically increasing with each additional correct answer, but can be framed as "percent correct" rule (PC), which shows reduction in score after an incorrect answer. Similarly, the two weighted scores that were used in the study, one was monotonically increasing (the reward points rule; RW), while the other was non-monotonically increasing (the "ability estimate" rule; AE). The results on TTs behavior indicated that the weighted scores encouraged TTs to challenge themselves to select more difficult items, however, risk aversion was observed for the non-monotonically increasing scores.;By and large, TTs choices were guided by their perceived ability, affected by the study's factors, both goal orientation and score feedback. Interaction was not found for the most part, except for effect on strategy (the dependency on previous item feedback) which was not affected in the learning oriented condition but was affected in the performance oriented group, indicating more attention to score feedback under the latter context. Cognitive involvement was also affected differently; score feedback (any of the four tested) increased involvement under a performance goal, but decreased it under a learning goal. Implication of the results are discussed.