Structural equation modeling estimates of reliability: A Monte Carlo study.

摘要

Coefficient alpha may be a biased estimate of reliability if the assumptions of essential tau equivalency and uncorrelated errors are violated. Alternative internal consistency estimates of reliability can be computed within a structural equation modeling (SEM) framework. SEM reliability estimation is reviewed for unidimensional items with and without correlated errors, and for multidimensional items, in particular, for items consistent with bifactor models. Two potential problems with SEM estimates of reliability are that they may be unstable when sample size is small and biased if the model is misspecified. A Monte Carlo study was conducted to investigate the quality of SEM estimates of reliability in comparison with coefficient alpha. Monte Carlo design factors included number of items, sample size, type of model used in the generation of the data, parameters for these models, and models for analyzing the generated data. The analysis models were underspecified, correctly specified, or overspecified with respect to the generation models. In total, the number of conditions created by the combination of all design factors was 252. SEM estimates of reliability and coefficient alpha were assessed by comparing their relative bias, efficiency, and precision computed on the 1000 data sets generated for each condition. SEM estimates of reliabilities were also evaluated taking into account the global fit of the analysis model. Results showed that in most conditions, SEM estimates yielded as good or better reliability than coefficient alpha. However, SEM estimates tended to be poorer if the generation model was misspecified, fit poorly, and were based on small sample sizes with low factor loadings. Based on the results, guidelines for the use of SEM estimates of reliability are suggested, and recommendations for future research on SEM estimates of reliability are offered.