Standardized visual predictive check versus visual predictive check for model evaluation

摘要

The visual predictive check (VPC) is a commonly used approach in model evaluation. However, it may not be feasible to conduct a VPC, or the results of a VPC could be misleading in certain situations. The objectives of the present study were to (1) examine the performance and applicability of the VPC and (2) propose the standardized visual predictive check (SVPC) as an alternative/complementary approach to the VPC. The difference between the SVPC and normalized prediction distribution error (npde) as visual tools for model evaluation is also discussed. The results of the simulation studies demonstrate that the VPC is not appropriate when stratification of covariate(s) in a model is difficult or arbitrary and may not be feasible when study design varies during a study/among participants. The SVPC addresses these issues by displaying the percentiles (Pi,j) of each participant's observations in the marginal distribution of the corresponding model-simulated endpoints as a function of time (or any covariate of interest) based on that participant's own design template. Since the calculation of Pi,j factors out subject-specific design features, the difference between observation and simulated values is only caused by misspecification of the structure model and/or inadequate estimation of random effect. Thus, the SVPC can be used in any situation.