Effects of anchor item content representation on the accuracy and precision of small sample linear test equating.

摘要

The purpose of this study was to empirically estimate the reduction in equating accuracy and precision that may occur when a non-content representative anchor item set is used to equate alternate test forms. For purposes of the study, hypothetical examination forms were constructed from archived item level data compiled from the administration of four subject area examinations administered as part of the Florida Teacher Certification Examinations (FTCE) licensure program, and one form of the Florida Educational Leadership Examination (FELE).; A total of 1,000 bootstrap samples for each experimental condition were drawn with replacement from the pseudo-populations formed from the archived data. Each experimental condition differed systematically across levels of content representation, anchor test length, sample size, and relative equivalence in the ability of the examinee groups. Levels of content representation were calculated by the use of existing item content classifications developed and validated during the construction of the test forms by subject matter experts.; Equating relationships between each operational and base form were estimated using the Angoff Model N method. The accuracy of each equating solution was estimated by calculation of two indices of statistical bias. The precision of equating was estimated by calculating the standard error of equating across bootstrap samples. The results of the research were presented using graphical displays and tabular data summaries.; Findings of the study indicated that the overall quality of equating was closely linked with the internal score reliabilities for the anchor and total test and the correlations between anchor test scores and total test unique components for each condition. Content representation of the anchor item set as defined by proportional concordance with the content categories delineated in the table of test specifications was not found to be consistently associated with the overall quality of equating. Reliability of the anchor and total test scores and the correlation between them may be more directly associated with the overall quality of common-item linear equating than the proportion of anchor items across test content categories.