The utility of a rigorous application of experimental design in the Air Force tactics development process.

摘要

In spite of a sound theoretical basis, there is a lack of rigorous application of Design of Experiments (DOE) in Air Force tactics testing. In evaluating DOE as a test methodology, this study had four distinct objectives: (a) assess the acceptance of DOE in the Air Force operational test community, (b) demonstrate the feasibility and benefits of DOE, (c) compare quantitatively various test design methodologies currently in use, and (d) present a step-by-step approach for tactics testing. A mixed research model was employed. The qualitative portion explained the methodologies currently in use, identified cultural and practical barriers to DOE, and framed the context for the quantitative section. The quantitative portion compared statistically test methodology effectiveness, accuracy, and efficiency. A stochastic simulation model of aerial combat was used to generate notional test results for the comparison. The study indicated a lack of rigorous application of DOE in tactics testing, and demonstrated that practical concerns about its use are largely unfounded. Moreover, a DOE test methodology produced more accurate outcomes using fewer test resource than other methodologies. The study recommends that operational testing be performed (a) as a multi-disciplined team of analysts and subject matter experts; and (b) by following a step-by-step approach based on DOE. A greater reliance upon DOE would represent a positive social change that saves costs and enhances the effectiveness of testing at a time when the military faces significant budget pressures, a drawdown in force structure, and increased global commitments. The same approach would benefit many similar organizations that perform experimentation characterized by complexity, high costs, and direct impacts on people's lives, such as the medical field, civil aviation, and consumer safety groups.