Measuring Success in Countering Terrorism: Problems and Pitfalls

摘要

Over-reliance on and misuse of statistical measurement not only has served to distort the intelligence product, but too often has corrupted the intelligence process as intelligence officers find themselves chasing the numbers with less time available to chase the hard but elusive information needed to advance the country's security interests. The mindset that produced the body count syndrome of the Vietnam War unfortunately is alive and well. It is part and parcel of our cultural baggage. As a consequence, it has caused us to fail to identify or misread critical trends and engage in practices that are transparently counterproductive. In seeking to measure success we look to measure things that are easy to quantify, but too often are off the mark, providing us only the illusion of accuracy and precision rather than a valid and accurate measure of meaningful progress. Statistical analysis as used by the government to assess terrorism and counter-terrorism efforts remains primitive and, too often, dangerously misleading. We measure what can easily be quantified rather than what is truly meaningful. We strive to capture extremely complex phenomena in a simple sound bite, reinforced by seemingly compelling but simplistic statistical comparisons and then wonder why our instant analysis has failed to comport with reality, leaving us embarrassed and scratching our heads. Numbers, as we use them, provide a false sense of objectivity, accuracy and precision, too often leaving the decision makers frustrated and angry. And, too often, leaving the public with the feeling that somehow they have been conned. In order to use our limited resources to best effect, we need to introduce concepts and analytic strategies that more accurately reflect the reality on the ground, enable us to better predict trends and more accurately assess the effectiveness of our efforts, our programs and our people.