NEPS technical report for science: Scaling results of starting cohort 4 in 11th grade

摘要

The National Educational Panel Study (NEPS) aims at investigating the development of competences across the whole life span and designs tests for assessing these different competence domains. In order to evaluate the quality of the competence tests, a wide range of analyses based on item response theory (IRT) have been performed. This paper describes the data on scientific literacy for starting cohort 4 in grade 11. Besides presenting descriptive statistics for the data, the scaling model applied to estimate competence scores and analyses performed to investigate the quality of the scale as well as the results of these analyses are also explained. The science test in grade 11 originally consisted of 29 multiple choice and complex multiple choice items and covers two knowledge domains as well as three different contexts. Five items had to be removed due to insufficient item quality. The test was administered to 4,417 students. A partial credit model was used for scaling the data. Item fit statistics, differential item functioning, Rasch-homogeneity, and the test’s dimensionality were evaluated to ensure the quality of the test. The results of the remaining test items illustrate good item fit values and measurement invariance across various subgroups. Moreover, the test showed a moderate reliability. The data shows that the assumption of unidimensionality of scientific literacy measured by this test seems adequate. Among the challenges of this test is the lack of very easy items. But overall, the results emphasize the good psychometric properties of the science test, thus supporting the estimation of reliable scientific literacy scores. In this paper, the data available in the Scientific Use File are described and the ConQuest-Syntax for scaling the data is provided. (IPN/Orig.)