An exploratory modeling survey of the trait structures of some existing language test datasets.

摘要

Recent developments in language testing have centered on implementation of multifaceted communicative language ability models. (The term 'facet' here refers to a theoretically-motivated trait or test design component, separate from statistically verified factors or dimensions.) At some language testing sites, multifaceted communicative tests are already being developed to make workaday decisions about student proficiency. This trend stands in contrast to older "unifaceted" language tests.;The present study examines existing language tests to see if multifaceted language models can be identified statistically. Twenty-one item-level datasets from nine batteries are modeled via unweighted least squares exploratory factor analysis (ULS EFA--using Muthen's 1987 LISCOMP program), and the interitem matrix of tetrachoric coefficients is then further analyzed by multidimensional scaling (MDS--using Young and Lewyckyj's ALSCAL program, available as a procedure in SAS, 1983). Determination of the number of factors/dimensions follows EFA and MDS procedures.;Analyses reveal the following striking conclusion: when more than one factor/dimension is found, the best factor labels are often pedestrian and spurious. For example: factors are uncovered which relate to the different days on which a test was administered, or to a large chunk of end-of-test missed items. There is one instance of a potential design facet or possibly a method facet: the listening section of the 1985 TOEFL. Generally, however, the study reaffirms the unitary factor statistical model for language ability.;This does not necessarily mean that language is unifaceted--i.e., that language ability should be viewed as a single theoretical mental trait. On the contrary, what this study may show is the need to supplement factor analysis and MDS for statistical verification of multiple language ability dimensions. The conclusion suggests that alternate facet-detection methodologies and epistemologies are needed. Additionally, various issues of sample size, number of items, and number of items per design facet may impinge on the results--resolution of these issues is also left for future research.