Testing a prototypic approach to scale reduction: Improving efficiency in assessment.

摘要

Test development and design are major issues in education, health care, and many other disciplines. The high stakes nature of testing requires high score reliability and validity. However, in many fields, respondent burden is also a serious concern. Modern test theory provides tools to address the need for shorter, more efficient assessment tools. This study assesses an approach to instrument reduction relying on structural equation modeling and item response theory (IRT) that should not only maintain the dimensionality and validity of the original long-form instrument but should have, at most, a negligible effect on reliability. The Teacher's Report Form was used to demonstrate this process. Behavior problem data collected using the Teacher's Report Form (Achenbach, 1991b) were available for a total of 856 children.; The purported factor structure of the TRF did not hold in the present data. Eighty-five of the original 120 TRF problem behavior items remained following an exploratory factor analysis that resulted in seven subscales: Aggressive Behavior, Attention Problems, Withdrawn, Anxious/Depressed, Somatic Complaints, Thought Problems, and Emotional Dysregulation. The latter subscale does not exist on the TRF. Two TRF subscales, Social Problems and Delinquent Behavior, were not found in the present data. Applying IRT analyses, 33 of the 85 items were identified for retention in the new screening instrument. Three additional items were included in the final instrument because of their clinical relevance. The final instrument consists of 36 items. The reliability of the new screener as a whole and for four of the final subscales was good to excellent. The reliability for the other two scales, Anxiety and Thought Problems, was not acceptable. Concurrent validity for the new instrument was, for the most part, good. The new instrument was able to predict maltreatment status equally as well as the TRF for the participants for whom this information was available. However, neither instrument could predict maltreatment status better than chance. The limitations of these results were discussed.