Factors affecting the accuracy of prediction models limit the comparison of rival prediction models when applied to separate data sets.

摘要

Two scenarios frequently appear in medical research when new medical prediction results are presented. In the first, a new marker is claimed to have higher prediction accuracy (eg, area under the receiver operating characteristic curve) than other markers evaluated in different data sets. In the second scenario, a new prediction model is argued to have higher predictive accuracy than do other prediction models, also evaluated in different data sets. These scenarios share the issue that predictive accuracies are being calculated and compared in different data sets. In other words, the predictions from the rival markers or rival prediction models are not head to head-not paired in the same neutral data set. These accuracy comparisons across data sets cause problems with interpretation.