Measuring what's missing: practical estimates of coverage for stochastic simulations

摘要

Stochastic sensitivity analyses rarely measure the extent to which realized simulations cover the search space. Rather, simulation lengths are typically chosen according to expert judgement. In response, this paper recommends a novel application of Good-Turing estimators of missing distributional mass. Using the United Nations Development Programme's Human Development Index, the empirical performance of such coverage metrics are compared to alternative measures of convergence. The former are advantageous - they provide probabilistic estimates of simulation coverage and permit calculation of strict bounds on estimates of pairwise dominance (for all possible weight vectors, how often country X dominates country Y).