The Meaning of Higher-Order Factors in Reflective-Measurement Models

摘要

Higher-order factor analysis is a widely used approach for analyzing the structure of a multidimensional test. Whenever first-order factors are correlated researchers are tempted to apply a higher-order factor model. But is this reasonable? What do the higher-order factors measure? What is their meaning? Willoughby, Holochwost, Blanton, and Blair (this issue) discuss this important issue for the measurement of executive functions. They came to the conclusion that formative measurement structure might be more appropriate than a reflective measurement structure with higher-order factors. Willoughby et al. refer to 4 decision rules for selecting a measurement model presented by Jarvis, MacKenzie and Podsakoff (2003). Whereas the rules of Jarvis et al. are based on plausibility arguments, we would like to go a step further and show that stochastic measurement theory offers a clear and well-defined theoretical basis for defining a measurement model. In our comment we will discuss how stochastic measurement theory can be used to identify conditions under which it is possible to define higher-order factors as random variables in a well-defined random experiment In particular, we will show that for defining second-order factors it is necessary to have a multilevel sampling process with respect to executive functions. We will argue that this random experiment has not been realized for the measurement of executive functions and that the assumption of second-order or even higher-order factors would not be reasonable for theoretical reasons. Finally, we will discuss the implications of this reasoning for the measurement of executive functions and other areas of measurement. We will start with the definition of first-order factors, and then we will discuss the necessary conditions for defining second-order factors.