Is this year's exam as demanding as last year's? Using a pilot method to evaluate the consistency of examination demands over time

摘要

Maintaining standards over time is a much debated topic in the context of national examinations in the UK. This study used a pilot method to compare the demands, over time, of two examination units testing administration. The method involved 15 experts revising a framework of demand types and making paired comparisons of examinations from different years and sessions (from June 2003 to June 2006) with respect to the types of demands in the framework and overall. Frequency data were used to ascertain the relative demands of examinations from different sessions. There was greater variation with some types of demands than others. However, overall demands appeared to be relatively stable from June 2003 to June 2004, whilst December 2004 exams were relatively more demanding and those after this were relatively lower in demands. The differences seem unlikely to have been sufficiently large to compromise the awarding process which acts to maintain standards. Participants completed a questionnaire evaluating the research method. The method was generally successful but with some room for improvement. This method has the potential to provide a means of evaluating the demands of examinations over time for other qualifications and hence to contribute to discussions about standards over time.