(1255)POSITIONAL EFFECT OF ITEM BLOCKS IN INTERNATIONALLARGE-SCALE ASSESSMENT CAUSED THE ITEM DIFFICULTYDIFFERENT BETWEEN TAIWAN AND THE USA

摘要

Many international large-scale educational survey studies, such as TIMSS, PIRLS and PISA, adoptedthe incomplete balanced booklet design to make sure that the estimated ability of examinees does notcovariate with the position of item blocks. By this incomplete design every two booklets contain somecommon items, and the ability scales of booklets drawn from the item response theory can be equatedacross booklets through these common items. However, the positional effect of item blocks existssomehow within a booklet, which helps in understanding the influential factors of the item difficulty. ForTIMSS 2007, a student spends two class-periods to complete a booklet, and each class-periodconsists of two item blocks. The current study thus explored the effects of the class-period (first vs.second class-period), the position of item block within a class-period (first vs. second position), andthe country (Taiwan vs. USA) on the item difficulty of TIMSS 2007 by treating them as facets of itemdifficulty in the generalized partial credit model. As the interaction suggests a practice effect in the firstclass-period where an item was less difficult at the second position than the first position, and a fatigueeffect in the second class-period where an item was more difficult at the second position than the firstposition across countries, the USA demonstrated a more serious fatigue effect than Taiwan.Consequently, the serious fatigue effect caused the items more difficult and lowered the abilityestimate of examinees of the USA in the end of testing.