SYSTEM AND METHOD FOR RATING AND SELECTING MODELS

摘要

Computer-implemented method and system are provided to identify superior models relative to a benchmark model in a step-wise fashion while reducing data snooping bias and increasing the test power. The data snooping bias may be reduced or avoided by controlling, in a step-wise fashion, a measure of error such as generalized family-wise error rate (FWER) and/or false discovery proportion (FDP). The test power of the method may be increased by relaxing the generalized FWER to tolerate more falsely rejected models and applying re-centering techniques to account for the inclusion of potentially “poor” models in the evaluation.