Development of Rainfall Forecast Performance Monitoring Criteria. Phase 1: Development of Methodology and Algorithms

摘要

Criteria for monitoring the quality of rainfall forecasts employed in support of flood warning are required to assess their reliability in use and to provide feedback aimed at providing an improved service. The rainfall forecasts of main concern here are the quantitative component of the Daily Weather Forecast, the Evening Update and the Heavy Rainfall Warning. These three products are produced by the Met Office as a service to the Environment Agency in support of their flood warning responsibilities.ududThis report, commissioned jointly by the Environment Agency and the Met Office, first reviews current methodology and practice in monitoring the performance of rainfall forecasts. The content, format and delivery mechanisms of each of the three forecast products are also reviewed and recommendations for revision made. The report proceeds to develop a framework for assessment, addressing issues such as selection of performance measures, choice of “ground truth”, and sources of comparative forecasts such as rainfall forecasts obtained directly from the Mesoscale Model and from the Nimrod radar-based product. New methods for assessing the accuracy of performance measures - as determined by a given rainfall forecast, ground-truth and comparative forecast dataset -are introduced.ududRainfall forecasts for case study storms are used to trial the assessment procedure employing a selection of performance measures. The case study storms were chosen by the Environment Agency to be of flooding interest to a number of its regions. Suitable ground truth available for assessment, including raingauge and Nimrod quality-controlled radar data, are identified and processed to a form suitable for application in the analysis. The analysis of the case study dataset is used to develop practical experience in the use of the assessment procedure leading to recommendations for operational implementation. These recommendations concern both the automated assessment of forecasts and the use of a PC tool with manual data-entry for assessing the Heavy Rainfall Warnings. The development of the PC tool features as an important operational output of the project.ududThe report concludes with a summary of the study, encompassing its main conclusions and recommendations. In particular, this points out the advantages of using a small and rather simple set of performance measures. The mean absolute error provides an easily understood and stable measure of the “typical size of error”, in the same units as the rainfall forecast. For a categorical measure of rainfall threshold exceedence, the Critical Success Index and False Alarm Rate provide a useful pairing that are widely used and easily understood. For assessing probability forecasts, the Continuous Brier Score provides a simple measure analogous in form to the mean absolute error. Measures of forecast bias are also included in the selected set of performance measures considered important.ud