The problem with expert predictions

 In Business decisions, Insights, Politics, Realizer Blog

The problem with Expert predictions

It makes sense that experts’ predictions are worth listening to, right? They must be able to read the current situation and use their experience and insight to predict what might happen next year – at least better than us non-experts.

Well, unfortunately not. A sizeable body of research has shown that experts, whether that be in politics, finance, health or even marriage guidance, are rarely better at predicting outcomes than if you used a simple formula.

Our favourite demonstration of this was a 20 year long study of expert political predictions by Philip Tetlock. He had experts look at a political situation and pick what would happen from 3 options – eg which of 3 candidates would win an election, or whether a situation would get better, worse or stay the same. He found unequivocally that the experts, whilst confident of their predictions, were no more accurate than if he’d thrown a dice, or asked a non-expert.

The reason for this is that experts often tend to latch onto one piece of knowledge, one aspect of a situation, and make their predictions mainly based on that. But the real world is more complicated.

A psychology professor called Paul Meehl looked at whether simple formulas could beat expert predictions. For example, he compared marriage guidance counsellors’ assessment of whether a couple would still be together in 5 years with a simple formula: could he better predict the couple’s chances from looking at their physical intimacy (number of times made love each month) and their argumentativeness (measured by the number of arguments they had each month). He found that if you did a formula of intimacy minus number of arguments, your predictions were better than the expert marriage counsellors. This result has been replicated across every domain it’s been tested in. Simple formulas beat expert predictions, every time.

The implication: see if you can simplify your predictions for 2015 based more on straightforward formulas and less on human judgement. Don’t listen to experts who tell you, for example, that the advertising market will rise 3% in 2015.

Read about Tetlock’s study of expert prediction
Watch a TED video on why not to trust experts