Confidence interval methods in discrete event computer simulation: Theoretical properties and practical recommendations.

摘要

Most of steady state simulation outputs are characterized by some degree of dependency between successive observations at different lags measured by the autocorrelation function. In such cases, classical statistical techniques based on independent, identical and normal random variables are not recommended in the construction of confidence intervals for steady state means. Such confidence intervals would cover the steady state mean with probability different from the nominal confidence level. For the last two decades, alternative confidence interval methods have been proposed for stationary simulation output processes. These methods offer different ways to estimate the variance of the sample mean with final objective of achieving coverages equal to the nominal confidence level. Each sample mean variance estimator depends on a number of different parameters and the sample size. In assessing the performance of the confidence interval methods, emphasis is necessarily placed on studying the actual properties of the methods in an empirical context rather than proving their mathematical properties. The testing process takes place in the context of an environment where certain statistical criteria, which measure the actual properties, are estimated through Monte Carlo methods on output processes from different types of simulation models. Over the past years, however, different testing environments have been used. Different methods have been tested on different output processes under different sample sizes and parameter values for the sample mean variance estimators. The diversity of the testing environments has made it difficult to select the most appropriate confidence interval method for certain types of output processes. Moreover, a catalogue of the properties of the confidence interval methods offers limited direct support to a simulation practitioner seeking to apply the methods to particular processes. Five confidence interval methods are considered in this thesis. Two of them were proposed in the last decade. The other three appeared in the literature in 1983 and 1984 and constitute the recent research objects for the statistical experts in simulation output analysis. First, for the case of small samples, theoretical properties are investigated for the bias of the corresponding sample mean variance estimators on AR(1) and AR(2) time series models and the delay in queue in the M/M/1 queueing system. Then an asymptotic comparison for these five methods is carried out. The special characteristic of the above three processes is that the 5th lag autocorrelation coefficient is given by known difference equations. Based on the asymptotic results and the properties of the sample mean variance estimators in small samples, several recommendations are given in making the following decisions: I) The selection of the most appropriate confidence interval method for certain types of simulation outputs. II) The determination of the best parameter values for the sample mean variance estimators so that the corresponding confidence interval methods achieve acceptable performances. III) The orientation of the future research in confidence interval estimation for steady state autocorrelated simulation outputs.