Modeling and Simulation to Adjust p Values in Presence of a Regression to the Mean Effect

摘要

Exploratory analysis methods, such as data mining, make it more common than ever to analyze data in a retrospective fashion, for example, to detect subgroups of interest. Such posthoc rnapproaches can easily be misleading, with an "effect" being found that is nothing more than an artifact of the data. One reason for such a finding—which is actually nonexistent—can be the well-known phenomenon of regression to the mean. It poses a challenge that, if ignored, can generate incorrect claims of significance.rnThis article summarizes the characteristics of regression to the mean and proposes an often easily implemented general approach to adjust such findings for potential confounding effects. The approach relies on the derivation of a reference distribution for calculated p values, based on modeling and simulation. The methodology allows for complex models to be used, because the simulations do not require that one works out the correlation structure of the parameter estimates analytically to adjust the regression to the mean effect. It also allows for a straightforward assessment of the goodness of the fit, following the idea of posterior predictive checks.rnWe illustrate the approach with the adjustment of the result from a recent study on patients suffering from Alzheimer's disease for which an unplanned subgroup selection was used after study completion.