Prepare items for Large Scale Computer Based Assessment: Case study for Teachers’ Certification on Basic Computer Skills

摘要

One of the main issues facing a Computer Based Assessment (CBA) system for large-scale examinations is the items’ exposure due to repeated e-exams that take place in different times or places. Administrators could reduce the side effects of item's exposure(possible leaks, copying in the examination process, etc)by enriching the item bank with items that assess the same knowledge, skills and attitudes in a different way. This paper presents the life-cycle of items used for teachers’ certification in basic computer skills in the frame of a national program that is carried out in Greece since 2003 (2003-2010). A methodology for preparing items for large scale assessment is introduced. The structure of items used and the roles of people involved are described. A process for preparing equivalent items (families) is introduced. The teachers’ behavior concerning equivalent groups of items (families) was explored based on candidates’ responses. Based on candidates’ responses that have been recorded since 2003, we refined a set of rules that must be taken into account by items’ authors or evaluators during the preparation of item bank for a large scale CBA system concerning basic IT skills.