The use of Monte Carlo sampling to study the performance of a sequential procedure for selecting the best bernoulli population

摘要

A closed adaptive sequential procedure for selecting the Bernoulli population with the largest "success" probability is described. Its use in various real-life settings including clinical trials is pointed out. Various optimal properties of the procedure had been proved analytically. However, it was desired in addition to assess quantitatively the "goodness" of the procedure for various performance characteristics of interest in applications. Exact calculations of these quantities could only be made at prohibitive cost. However, Monte Carlo sampling provided an easily implemented method of estimating these quantities at modest cost. In this presentation we describe the Monte Carlo method that was used to estimate the performance characteristics of interest. The results obtained are given in detail. They proved to be very illuminating, and provided strong justification for using the procedure in practical settings.

The purpose of this presentation is to describe the important role that Monte Carlo sampling played in studying the performance of a closed adaptive sequential procedure for selecting that one of k Bernoulli populations which has the largest "success" probability. The procedure in question was proposed by Kulkarni (1981); subsequent articles, Bechhofer and Kulkarni (1982a), Bechhofer and Kulkarni (1982b), Bechhofer and Frisardi (1983) [based in part on Frisardi (1982)], and Kulkarni and Jennison (1983) studied in depth various important properties (including optimality properties) which would bear on the usefulness of the procedure in real-life settings. (See below.) All but the Bechhofer-Frisardi article were studies in which the various optimality results and related ones were proved analytically. However, it is only part of the picture to know that a procedure is optimal; of great practical importance is the quantitative assessment of how much better this optimal procedure is than any other competing procedure such as, for example, the commonly used single-stage procedure of Sobel and Huyett (1957). It sometimes is possible to make exact calculations of these gains, but even when it is possible to do so, the computing costs can become prohibitive as the various input parameters are varied over wide ranges. However, such studies can often be made very easily and at modest cost when Monte Carlo sampling is employed.

The main focus of this presentation will be the results described in the Bechhofer-Frisardi article. The competing procedures, the sequential and the single-stage procedures referred to above, will be described in detail, and the performance characteristics of interest will be emphasized. The procedures would appear to be applicable, e.g., in clinical trials in medicine, where the object of the experiment is to find the treatment which produces the highest proportion of "successes" (cures). In such experiments it is important not only to guarantee a high probability of selecting the best treatment but also to accomplish this using the minimum number of "patients"on the average, and minimizing the average number of "patients" given the inferior treatment(s).

The reader who may be interested in the literature on Bernoulli selection procedures will find a large number of references in Bechhofer and Kulkarni (1982a). For early work on ranking and selection procedures see Bechhofer (1954) and Gupta (1965); specialized books on this subject include Gibbons, Olkin and Sobel (1977) (see also the review by Bechhofer (1980)) and Gupta and Panchapakesan (1979); a comprehensive list of references is contained in Dudewicz and Kuo (1982).

Some of the author's previous experience with Monte Carlo sampling is contained in Gershefski (1958), Ramberg (1966), and Bechhofer, Kiefer and Sobel (1968) (see, in particular, Chapter 18).