A framework for evaluating technology readiness, system quality, and program performance of U.S. DoD acquisitions .

摘要

The purpose of this study is to characterize the current state of the United States Department of Defense (DoD) Technology Readiness Assessment (TRA) practice and enabling engineering activities within the DoD acquisition programs. Further, this study examines the extent of correlation between three constructs: 1) implementation of TRAs and enabling engineering activities, 2) military system quality objectives, and 3) acquisition performance goals measured by cost, schedule, customer satisfaction, and productivity. Although the DoD is a world leader in the acquisition of advanced weapon systems, over the past several decades the Government Accountability Office (GAO) has consistently reported that the DoD acquisition programs are experiencing difficulties as a result of advancing through the various stages of the acquisition lifecycle with insufficiently mature technologies among other constructs such as unstable design, lack of manufacturing maturity, and knowledge gaps. By means of policies, initiatives, and best-practices, the U.S. DoD has perpetually invested in refining its acquisition policies in an effort to curtail these risks. In recent years DoD has imposed assessment of technology maturity through the implementation of Technology Readiness Assessment (TRA) on all its acquisition programs in order to mitigate these risks by capturing the right knowledge at the right time. However, there are a limited number of scholarly studies that evaluate the extent to which defense acquisition programs are implementing DoD mandates and best-practices that enable TRAs. Further, the correlation between TRAs, TRA enabling engineering activities, and military system quality has not been examined in a quantitative scholarly manner. In this study, a framework was developed to measure the relationships between DoD's TRA and enabling engineering activities, and U.S. military system quality. The framework was developed in conjunction with Subject Matter Experts (SMEs) from DoD, Intelligence Community (IC), and Institute of Defense Analysis (IDA), and empirically examined using a sample of 223 engineers, program managers, and executives from various U.S DoD acquisition programs. In order to capture the attitudes and experiences of acquisition engineers and Program Managers (PMs) in regard to technology maturity assessment, additional data was collected by structured interviews with eleven senior-ranking acquisition executives. Further, the data gathered from the interview subjects provides insight into how DoD organizations and contractors comply or fail to comply with TRA and interrelated tenets. The results of this study conclude that although formal Milestones B and C TRA emerged to not be significant in predicting the quality of U.S. military systems and performance of acquisition programs, various engineering activities that are examined during the TRA process were significant. Therefore it can be inferred that formal Milestones B and C TRAs may be of little value if due diligence is applied during acquisition lifecycle by adequately implementing engineering activities that reduce technical and programmatic risks.