A very interesting article by Gregory Treverton (who I know from his distinction between puzzles and mysteries in forecasting) asks "What should we expect of our spies?" It's available to subscribers, so you may or may not be able to read the whole thing, but a couple points jumped out at me.
The main question he takes on is,
"What should people expect of their intelligence agencies?" Not "what would they like?"; for policymakers would like perfect prescience, if not omniscience. They know that they can have neither.
The first is his distinction between different types of tasks that intelligence analysts are called upon to undertake, and the danger of confusing one with the other. There are demands for both actionable intelligence or answers to specific questions– i.e., the answers to puzzles– and a broader awareness of large trends or potential disruptions– in other words, mysteries– and the pursuit of the two don't sit together well.
When the Soviet Union would collapse was a mystery, not a puzzle. No one could know the answer: it depended. It was contingent. Puzzles are a very different kind of intelligence problem. They have an answer, but we may not know it. Many of the intelligence successes of the cold war were puzzle-solving about a very secretive foe: were there Soviet missiles in Cuba? How many warheads did the Soviet SS-18 missile carry?
The second interesting point is that at the National Intelligence Council,
I came to think that, for all the technology, strategic analysis was best done in person. I came to think that our real products weren't those papers, the NIEs. Rather they were the NIOs, the National Intelligence Officers-the experts, not papers. We all think we can absorb information more efficiently by reading, but my advice to my policy colleagues was to give intelligence officers some face time. If policymakers ask for a paper, what they get will inevitably be 60 degrees off the target. In 20 minutes, though, the intelligence officers can sharpen the question, and the policy official can calibrate the expertise of the analyst.
This is a nice reminder how just how much our thinking about the future is a product of our current concerns– that intelligence "is about creating and adjusting stories," as Treverton puts it, and thus "stories not imagined by policy are not likely to be answered or developed by intelligence"– and how much knowledge about the future is irreducibly craft and tacit knowledge, not formal knowledge. Sure, you can produce a report, this line of argument goes, but that product will by its nature be misleading.