Acquisition Program Problem Detection Using Text Mining Methods

摘要

This research provides program analysts and Department of Defense leadership with an approach to identifying cost problems in acquisition contracts. Specifically, we test the efficiency of algorithms to detect unusual changes in acquisition programs' cost estimates using text mining techniques. Currently, the government obtains Contract Performance Reports (CPRs) on a monthly basis as status updates of acquisition programs. These CPRs include monthly data on the dollar amounts spent compared to what has been allocated to be spent (Format 1 data). In addition, the CPRs include a written portion (Format 5) that qualitatively documents the occurrences of the past month. Traditional Earned Value Management (EVM) analysis uses the Format 1 data to detect potential cost problems in acquisition contracts. However, the written portion of CPRs is usually used only to reference something in case a problem is detected in the dollar amount (Format 1) portions. We seek to answer the following questions: (1) Can we accurately quantify qualitative textual data through text mining methods; (2) Can we determine a relationship between the text mining results and Earned Value data; (3) Can we use the results from text mining methods to predict changes in a contractor's Estimate at Complete (EAC); and (4) If we can predict changes, how accurate are these predictions. To answer these questions, we use text mining, an approach that counts words and phrases and uses those counts to determine predictability, in this case to determine whether or not the acquisition program may experience a major problem (5% change in expected costs) in the future. If we can successfully use qualitative data (Format 5) to predict EAC changes and potential cost overruns, we will have accomplished something never before done in the DoD cost field.