Malcolm Galdwell has a great article in the latest New Yorker on "Enron, intelligence, and the perils of too much information." It's about contrasting types of complex problems, and it holds a useful lesson for how to think about the future.
The national-security expert Gregory Treverton has famously made a distinction between puzzles and mysteries. Osama bin Laden’s whereabouts are a puzzle. We can’t find him because we don’t have enough information. The key to the puzzle will probably come from someone close to bin Laden, and until we can find that source bin Laden will remain at large.
The problem of what would happen in Iraq after the toppling of Saddam Hussein was, by contrast, a mystery. It wasn’t a question that had a simple, factual answer. Mysteries require judgments and the assessment of uncertainty, and the hard part is not that we have too little information but that we have too much.
In fields as diverse as oncology and military intelligence, as the ability to gather information and volume of data have gone up, levels of certainty have gone down. What used to be treated as puzzles must now be treated as mysteries. There used to be one test for prostate cancer, now there are many. They provide earlier and earlier signs of cancer, but (so Gladwell argues) they are also more ambiguous.
How does this connect to the future?
Our default assumption about the future is that it's a puzzle. There is A Single Future, just like a single location where Osama bin Laden is hiding, and with enough information, we can solve it. But in reality, the future isn't a puzzle; it's a mystery. We can assemble vast amounts of information that offer clues about the future, but that information is going to be full of contradictions, mixed signals, and noise. Piling on even more facts won't make the future easier to divine; it'll make seeing the future harder.
Why is this so? Because there is no Single Future that futurists should be looking for. There are many possible futures, and the job of the futurist is to sort out which are more likely, and to help people see the contingency and opportunity in those futures. If fortunetellers traffic in knowledge of inevitabilities– in knowing exactly what is going to happen and when– the end-point of futurists' work should be a better knowledge of the contingency that's hard-wired into the future.