For a while I've been looking at the use of scenarios in the intelligence community, and I've been interested in is the way scenarios are used by people in very high-stakes situations. Most of the time scenarios are used to illustrate possible futures that could unfold over years, but intelligence people have to think about threats that are very hard to detect, originate in hard-to-study parts of the world, and have to be identified and dealt with quickly. We tend to think of this kind of identification or pattern-recognition as something that has to be done by computers running sophisticated programs, so how can scenarios help in those environments? Shane Harris' new book The Watchers fills in a piece of the puzzle for me:
Jeff Jonas… criticizes data mining for the "terrorist discovery problem." Jeff's essential point is that in order for data mining to work, you have to have a target. You have to have a known "bad guy," or a suspect, and then go to work mining data on him. The technology doesn't help you when you're searching for an unknown bad guy in the vast, noisy data cloud, without much of an idea where to start.
But here's where Poindexter took issue with Jonas. He acknowledged that data mining uses statistical analysis to find connections. And since terrorist attacks are statistically so rare, data mining isn't very useful when you don't know who the terrorist is. So Poindexter wanted to use "red teams," groups of terrorism experts who'd devise likely attack scenarios and then identify the patterns of activity that terrorists must engage in before they strike. TIA would then hunt in the data for evidence of those patterns.
There are two interesting things here for me. The first is that the system Poindexter proposed recognizes that the pattern-identification is still fundamentally a human problem: it requires creativity and a certain deviousness of mind to connect the dots (and that's not made easier when you have billions of dots, some of which aren't really dots but just noise). But the second is that, in a way, this is a pretty familiar use of scenarios: you use them to help identify what you should be looking for. It's just that the monitoring happens at a far larger than anything most organizations or individuals are accustomed to.