Evaluating uncertainty estimates in hydrologic models: borrowing measures from the forecast verification community

摘要

The hydrologic community is generally moving towards the use ofprobabilistic estimates of streamflow, primarily through the implementationof Ensemble Streamflow Prediction (ESP) systems, ensemble data assimilationmethods, or multi-modeling platforms. However, evaluation of probabilisticoutputs has not necessarily kept pace with ensemble generation. Much of themodeling community is still performing model evaluation using standarddeterministic measures, such as error, correlation, or bias, typicallyapplied to the ensemble mean or median. Probabilistic forecast verificationmethods have been well developed, particularly in the atmospheric sciences,yet few have been adopted for evaluating uncertainty estimates in hydrologicmodel simulations. In the current paper, we overview existing probabilisticforecast verification methods and apply the methods to evaluate and comparemodel ensembles produced from two different parameter uncertainty estimationmethods: the Generalized Uncertainty Likelihood Estimator (GLUE), and theShuffle Complex Evolution Metropolis (SCEM). Model ensembles are generatedfor the National Weather Service SACramento Soil Moisture Accounting(SAC-SMA) model for 12 forecast basins located in the Southeastern UnitedStates. We evaluate the model ensembles using relevant metrics in thefollowing categories: distribution, correlation, accuracy, conditionalstatistics, and categorical statistics. We show that the presentedprobabilistic metrics are easily adapted to model simulation ensembles andprovide a robust analysis of model performance associated with parameteruncertainty. Application of these methods requires no information inaddition to what is already available as part of traditional modelvalidation methodology and considers the entire ensemble or uncertaintyrange in the approach.