A correction model for differences in the sample compositions: the degree of comparability as a function of age and schooling

摘要

Background Since the early days of international large-scale assessments, an overarching aim has been to use the world as an educational laboratory so countries can learn from one another and develop educational systems further. Cross-sectional comparisons across countries as well as trend studies derive from the assumption that there are comparable groups of students in the respective samples. But neither age-based nor grade-based sampling strategies can achieve balanced samples in terms of both age and schooling. How should such differences in the sample compositions be dealt with? Methods We discuss the comparability of the samples as a function of differences in terms of age and schooling. To improve the comparability of such samples, we developed a correction model that adjusts country scores, which we evaluate here with data from different IEA (International Association for the Evaluation of Educational Achievement) studies on reading at the end of primary school. Results Our study demonstrates that ignoring differences in age and schooling confounds league tables and hides actual trends. In other words, cross-sectional comparisons across countries as well as trends within countries are affected by differences in the sample composition. The correction model adjusts for such differences and increases the comparability across countries and studies. Conclusions Researchers who use the data from international comparative studies for secondary analyses should be aware of the limited comparability of the samples. The proposed correction model provides a simple approach to improve comparability and makes the complex information from international comparisons more accessible.