Why economic predictions are a shot in the dark
In a speech Wednesday, Charles Bean, the deputy governor of the Bank of England, admitted the bank had failed to see the Great Recession coming. But the same was true of virtually all other forecasters, he said. And it would happen again. "One should not expect to be able to predict the timing and scale of these sorts of events with any precision," Bean said.
Given the evidence, every forecaster should be as humble as Charles Bean. But they're not. When I watch business TV, or read commentary in newspapers, or browse the best-sellers at book stores, I see a parade of fast-talking experts who know what's coming. They're certain of it.
And that's a big problem.
In the late 1980s, University of California psychologist Philip Tetlock launched the most comprehensive analysis of expert predictions ever conducted. After assembling a group of almost 300 experts -- economists, political scientists, journalists -- Tetlock peppered them with questions. Will the inflation rate rise or fall? What about economic growth, the price of oil, the stock market? Will there be an election? Who will win? All the sort of stuff we see in the media. In total, over many years, Tetlock gathered an astonishing 27,000 predictions.
Time passed. Accuracy was checked. Data were crunched. And Tetlock discovered the average expert was about as accurate as that famous symbol of random guessing -- the dart-throwing chimpanzee.
Some experts did worse than average. Others did better, although they were still miles from perfect. What distinguished the two groups? It made no difference whether they were liberal or conservative. Nor did it matter whether they were optimists or pessimists, whether they had PhDs, access to classified information, or lots of other factors you might think would make a difference.
What made all the difference was the experts' style of thinking.
Experts who did better than average gathered information from many sources, looked at issues with more than one analytical lens, readily admitted mistakes, and were comfortable with complexity and uncertainty. They talked about possibilities and probabilities. They often said "maybe." And they were humble about their ability to see into the future.
They were a lot like Charles Bean.
Experts who did worse than average? Just the opposite. One analytical lens. Drew on fewer information sources. Reluctant to admit mistakes. And they disliked complexity and uncertainty. They didn't say "maybe." They knew exactly what was going to happen. They were certain of it.
Yes, the sort of expert who would have been beaten by a dart-throwing chimpanzee was identical to the sort of expert who dominates the media. Not surprisingly, Philip Tetlock found an inverse correlation in his data: the more famous an expert was, the less accurate he was.
But the problem isn't the media. It's us.
People have a profound psychological aversion to uncertainty. We need to believe we know what is coming. We want to believe. And so we find the expert who tells a clear and simple story with a crisp conclusion to be far more compelling than the expert who says things are complex and uncertain and we can only be sure what will happen when it happens.
Ultimately, it's not the cocksure experts who fool us. We fool ourselves.
-- By Dan Gardner, author of "Future Babble"