Secondary analysis of large-scale international datasets in the service of national education policy evaluation: The case of PISA Australia

摘要

National educational policy analysis and evaluation is a complex endeavour that would seem to demand empirical data gathering efforts that are of appropriate scale and high quality. However, mounting such data-gathering efforts can be resource and time-intensive. As an alternative strategy, this presentation describes the secondary analysis of an existing large-scale dataset that potentially adds value to educational policy evaluation. In particular, as a member of the Organisation for Economic Cooperation and Development (OECD), Australia participates in the Programme for International Student Assessment (PISA) that every few years assesses the educational attainment of 15-year old students in the core learning areas of reading, maths, science and problem solving. PISA datasets are housed and managed by the Australian Centre for Educational Research (ACER) and it is this dataset that is the subject of our secondary analysis here.udud For the current policy question, Australia’s new Commonwealth government has begun consideration of applying a so-called “socioeconomic status (SES) model” to public school funding. We suggest that the secondary analysis of extant large-scale datasets can provide important input to the discussion of school funding policies by shedding light on previously obscured or possibly unexamined relationships. For example, it is already well established in the educational research literature that the socioeconomic status of individual students is strongly associated with educational attainment as measured by standardized assessment systems, whether local, national or international. In addition, various international studies have shown that the aggregated socioeconomic profile of a school is also positively associated with students’ academic attainment. ududHowever, less is known about the nature of these relationships when both individual student and school socioeconomic status are disaggregated. To uncover these finer-grained associations, we subjected Australia’s 2003 PISA dataset to secondary analysis to better understand the reading, mathematics and science achievement of secondary school students from different SES backgrounds, across a variety of school SES strata. This finer-grained secondary analysis shows that increases in the aggregated SES of a school are consistently and strongly associated with increases in students’ academic performance, and that this relationship holds for all students regardless of their individual SES. In the Australian case, the aggregated socio-economic profile of the school matters greatly in terms of academic performance. We conclude the presentation with a discussion of the implications of these findings for Australia’s federal school funding policies with particular attention given to the influence of school composition on student attainment.