Where are we going?

How even dart-throwing chimpanzees can learn to forecast political and economic events

With vivid memories of Brexit, the US elections and other unexpected events, it might seem an unlikely claim but forecasting is a skill some people possess. And it can be learned. “Forecasting is not a ‘you have it or you don’t’ talent. It is a skill that can be cultivated,” says Philip Tetlock.

A professor of psychology and political science at the University of Pennsylvania, Tetlock and his research (and life) partner Barbara Mellers commenced the Good Judgment Project (GJP). Over the project’s four-year duration, more than 20,000 “intellectually curious laypeople” tried to forecast the future while Tetlock, Mellers and a multidisciplinary team at the universities of Pennsylvania and California rigorously tested their team’s projections for accuracy, improving them every year and thereby unveiling what makes forecasts accurate.

Tetlock’s team – and four others like it – competed in a forecasting tournament held by the US intelligence agency Intelligence Advanced Research Projects Activity (IARPA). Reporting to the Office of the Director of National Intelligence, IARPA is tasked with furthering research that strengthens US intelligence agencies like the CIA, FBI and the NSA, to name but a few.

Like talk-show pundits in the public sphere, intelligence analysts spend time and money forecasting the future, but they hardly assess how good their forecasting actually is. To change that, IARPA created a four-year tournament, asking participating teams to submit their forecasts on a total of 500 questions about world affairs every day at 9 am from September to June. “After two years, GJP was doing so much better than its academic competitors that IARPA dropped the other teams,” Tetlock reports in Superforecasting: The Art and Science of Prediction.

THINKING RIGHT

Tetlock reaches two conclusions. One, short-term foresight is real. “Some people have a real, measurable skill at judging how high-stakes events are likely to unfold three months, six months, a year or a year and a half in advance.” His second insight is that forecasting is no particular gift but “the product of particular ways of thinking, of gathering information, of updating beliefs. These habits of thought can be learned and cultivated by any intelligent, thoughtful, determined person.” What matters is how superforecasters think. “Broadly speaking, superforecasting demands thinking that is open minded, careful, curious and, above all, self-critical,” Tetlock and his co-author Dan Gardner write.

Tetlock’s recipe for understanding the future is to “start by finding and retaining good forecasters, provide them with cognitive-debiasing training, then supply the right environment to generate their forecasts through collaborative teamwork and then apply statistical methods to draw out the wisdom of the crowd,” he tells PROJECT M.

There is more to this than academic or intelligence interest. Understanding the future is crucial to economic growth, as research by Scott Baker (Northwestern University), Nick Bloom (Stanford) and Steven Davis (University of Chicago) shows. With an index of economic-policy uncertainty that goes back to the, they show that high levels of policy unpredictability go with lower business investment and weaker economic growth.

GO PROBABILISTIC

Tetlock’s findings have been warmly received by “fellow Bayesians in forward-looking financial firms,” he says. “Like superforecasters, successful financial professionals tend to be open minded and are comfortable with probabilistic thinking.”

Warren Hatch is one example. A former hedge fund manager, he is now with Good Judgment Inc., the commercial spin-off that followed from the academic success. Hatch recalls that “I have learned to better gauge how confident I am in my investment forecasts. For instance, I have a target and then forecast the probability that the target will be hit within some band.” Good Judgment Inc. also collaborates with The Economist on a public forecasting tournament with questions posted at the Good Judgment Open.

Yet Tetlock, who after earlier research on assessing the accuracy of predictions between and made the infamous comment on most experts being as accurate as “dart-throwing chimps,” is well aware of the limits of forecasting. “I remain in awe at the complexity of human decision making. But navigating life’s challenges is made a little easier for all us through probabilistic thinking.”

Where are we going?

How even dart-throwing chimpanzees can learn to forecast political and economic events

Where are we going?

How even dart-throwing chimpanzees can learn to forecast political and economic events

Melania Trump’s dress predicts a volatile time

Economics: evidence isn’t everything

Decreasing sense of smell can predict death

Ditch the demographic gloom: Europe isn’t dying, yet