Probable more likely than probably

What kind of probability are people talking about when they say something is "highly likely" or has "almost no chance"? The chart below, created by Reddit user zonination, visualizes the responses of 46 other Reddit users to "What probability would you assign to the phase: <phrase>" for various statements of probability. Each set of responses has been converted to a kernel destiny estimate and presented as a joyplot using R. 

Somewhat surprisingly, the results from the Redditors hew quite closely to a similar study of 23 NATO intelligence officers in 2007 the 60's or 70's (the timeline of the study isn't clear — see comment below) . In that study, the officers — who were accustomed to reading intelligence reports with assertions of likelihood — were giving a similar task with the same descriptions of probability. The results, here presented as a dotplot, are quite similar.

For details on the analysis of the Redditors, including the data and R code behind the joyplot chart, check out the Github repository linked below.

Github (zonination): Perceptions of Probability and Numbers

Comments

[Charles Twardy]

Note: The survey of 23 NATO officers is not from 2007, but at least 30 years earlier, as you might guess by the hand-generated plot. The 2007 date is just the posting date for when cia.gov made Richards Heuer's 1999 Psychology of Intelligence Analysis available online. Most of Heuer's chapters were written by 1986. The NATO plot is republished from the 1977 Handbook for Decision Analysis (Barclay et al., see Heuer's footnote 141).

I no longer have a copy of that report to look up the date of the NATO survey, but it would be between 1964 (when Sherman Kent wrote his "Words of Estimative Probability" and 1977 (when the handbook was published).

[David Smith]

Thanks for the useful context, Charles! I've updated the post above.