Empirical Study of Re-sampling Techniques as a Method for Improving Error Estimates in Split-plot Designs

摘要

For any acquisition program, whether Department of Defense (DOD) or industry related, the primary driving factor behind the success of a program is whether or not the program remains within budget, stays on schedule and meets the defined performance requirements. If any of these three criteria are not met, the program manager may need to make challenging decisions. Typically, if the program is expected to not stay within budget or is expected to be delayed for one reason or another, the program manager will tend to limit areas of testing in order to meet these criteria. The result tends to be a reduction in the test budget and/or a shortening in the test timeline, both of which are already lean. The T&E community needs new test methodologies to test systems and gain insight on whether a system meets performance standards, within the budget and timeline constraints. In particular, both fundamental and advanced aspects of experimental design need to be adapted. The use of experiential design within DOD has continued to grow because of the needed adaptation. Many different types of experiments have been used. An experimental design that is often needed is one that involves a restricted randomization design such as a split-plot design. Split-plot designs arise when specific factors are difficult (or impossible) to vary, a frequent occurrence within the T&E community. However, split-plot designs have limitations on the estimation of the whole plot (hard to change) and sub plot (easier to change) errors without the conduct of a sufficient number of replications for the design. Within the timeline constraints for particular programs, sufficient replications are difficult, even impossible to complete. The inability to conduct the sufficient replications often lead to models that lack precision in error estimation and thus imprecision in corresponding conclusions.