I love you with 95% certainty

Image from "Safety risk and management"

A few days ago, "Nature" published a paper titled "Twenty tips for interpreting scientific claims", by William Sutherland, David Spiegelhalter and Mark Burgman. The paper is described as dedicated to "helping non-scientists" in understanding scientific claims, especially in relation to the climate change problem. The authors claim that "the immediate priority is to improve policy-makers' understanding of the imperfect nature of science. The essential skills are to be able to intelligently interrogate experts and advisers, and to understand the quality, limitations and biases of evidence." (emphasis mine)

It is an appreciable effort, but I think it totally misses the point.  The first problem is that the list is understandable (actually, obvious) for scientists, but not at all for non-scientists. Let me report one of these 20 points as an example

Significance is significant. Expressed as P, statistical significance is a measure of how likely a result is to occur by chance. Thus P = 0.01 means there is a 1-in-100 probability that what looks like an effect of the treatment could have occurred randomly, and in truth there was no effect at all. Typically, scientists report results as significant when the P-value of the test is less than 0.05 (1 in 20).


I don't know what kind of policy-makers you deal with, normally, but those I am acquainted with won't make it beyond the first point. But, apart from this problem, I think that the paper misses the essence of the debate on climate change. Many scientists still operate on the basis of the "information deficit model"; that is, they assume that people will make the right choices when correctly informed. Unfortunately, this is not the way the real world works.

As Dan Kahan has amply shown many times, in issues such as climate change people first take a position based on their pre-conceived ideas, then look for facts that support their position. We all know that science, as every human endeavor, is subjected to uncertainties but, given the way people behave, it is simply suicidal to emphasize "the imperfect nature of science" in the debate, as the authors do. It means giving ammunition to those who have been playing the uncertainty card against science. It means confusing those who have been honestly trying to understand the problem of climate change. It means forgetting that people don't just have a mind, but also a heart and that if you want to move them to action, then you must win their hearts and minds!

Think about that: would you tell to your sweetheart "I love you with 95% certainty"? Not even a scientist would.


______________________________________________________


 Here is the list of the 20 points of the paper by Sutherland et al. The full article on Nature is here.

1. Differences and chance cause variation.
2. No measurement is exact.
3. Bias is rife.
4. Bigger is usually better for sample size.
5. Correlation does not imply causation. 
6. Regression to the mean can mislead.
7. Extrapolating beyond the data is risky.
8. Beware the base-rate fallacy.
9. Controls are important. 
10. Randomization avoids bias.
11. Seek replication, not pseudoreplication.
12. Scientists are human.
13. Significance is significant. 
14. Separate no effect from non-significance.
15. Effect size matters.
16. Study relevance limits generalizations.
17. Feelings influence risk perception.
18. Dependencies change the risks.
19. Data can be dredged or cherry picked.
20. Extreme measurements may mislead.