Estimation of Incremental Validity in an Employment Setting: A Review of RelevantLiterature and Some Research Design Considerations

摘要

Improved validity coefficients have been reported for retest ASYAB scores. Twohypotheses were made to explain this effect: Temporal Contiguity (TC), which posits a change in true scores over time, and Better Ability Estimate (BAE), which posits a reduction in error of measurement but no true score change. This report critically reviews literature relevant to the evaluation of these two hypotheses and proposes a research design to investigate the hypotheses. The evaluation of the viability of either hypothesis is viewed as partly dependent on the type and number of criteria chosen to be predicted, and the timing of data collection on the criteria. The probability of true score changes occurring is viewed as varying within and across predictor domains (cognitive, perceptual/psychomotor, biodata, personality, vocational interests). The probability of BAE explaining validity increases is viewed as partly dependent on the nature of interventions to reduce error of measurement. A research design to investigate the two hypotheses directly is described and power analyses are provided for the design. Finally, it is suggested that more than one study will be necessary to provide firm conclusions about the TC versus BAE hypothesis, and it might be more prudent to design incremental validity research to avoid confounding with these issues.