Strategies of Communication on Climate Change

Sunday, December 1, 2013

I love you with 95% certainty

Image from "Safety risk and management"


A few days ago, "Nature" published a paper titled "Twenty tips for interpreting scientific claims", by William Sutherland, David Spiegelhalter and Mark Burgman. The paper is described as dedicated to "helping non-scientists" in understanding scientific claims, especially in relation to the climate change problem. The authors claim that "the immediate priority is to improve policy-makers' understanding of the imperfect nature of science. The essential skills are to be able to intelligently interrogate experts and advisers, and to understand the quality, limitations and biases of evidence." (emphasis mine)

It is an appreciable effort, but I think it totally misses the point.  The first problem is that the list is understandable (actually, obvious) for scientists, but not at all for non-scientists. Let me report one of these 20 points as an example

Significance is significant. Expressed as P, statistical significance is a measure of how likely a result is to occur by chance. Thus P = 0.01 means there is a 1-in-100 probability that what looks like an effect of the treatment could have occurred randomly, and in truth there was no effect at all. Typically, scientists report results as significant when the P-value of the test is less than 0.05 (1 in 20).


I don't know what kind of policy-makers you deal with, normally, but those I am acquainted with won't make it beyond the first point. But, apart from this problem, I think that the paper misses the essence of the debate on climate change. Many scientists still operate on the basis of the "information deficit model"; that is, they assume that people will make the right choices when correctly informed. Unfortunately, this is not the way the real world works.

As Dan Kahan has amply shown many times, in issues such as climate change people first take a position based on their pre-conceived ideas, then look for facts that support their position. We all know that science, as every human endeavor, is subjected to uncertainties but, given the way people behave, it is simply suicidal to emphasize "the imperfect nature of science" in the debate, as the authors do. It means giving ammunition to those who have been playing the uncertainty card against science. It means confusing those who have been honestly trying to understand the problem of climate change. It means forgetting that people don't just have a mind, but also a heart and that if you want to move them to action, then you must win their hearts and minds!

Think about that: would you tell to your sweetheart "I love you with 95% certainty"? Not even a scientist would.


______________________________________________________


 Here is the list of the 20 points of the paper by Sutherland et al. The full article on Nature is here.

  1. Differences and chance cause variation.
  2. No measurement is exact.
  3. Bias is rife.
  4. Bigger is usually better for sample size.
  5. Correlation does not imply causation. 
  6. Regression to the mean can mislead.
  7. Extrapolating beyond the data is risky.
  8. Beware the base-rate fallacy.
  9. Controls are important. 
  10. Randomization avoids bias.
  11. Seek replication, not pseudoreplication.
  12. Scientists are human.
  13. Significance is significant. 
  14. Separate no effect from non-significance.
  15. Effect size matters.
  16. Study relevance limits generalizations.
  17. Feelings influence risk perception.
  18. Dependencies change the risks.
  19. Data can be dredged or cherry picked.
  20. Extreme measurements may mislead. 

8 comments:

  1. Ouch. Deniers will point to this Nature article as "proving" that science about AGW is inconclusive.

    ReplyDelete
  2. I think this list will be dredged or cherry picked, without understanding, and used to mislead. You're quite right

    ReplyDelete
  3. Ugo,

    Did you not see this article in science earlier this week?

    Some mad scientists are trying to put a 95% uncertainty on true love! :
    http://www.bbc.co.uk/news/science-environment-25129117

    ReplyDelete
    Replies
    1. Don't they say that love is a chemical reaction?

      Delete
    2. In some way that probably is true, but that doesn't mean pharmaceutical companies should ever sell 'love pills'.

      Delete
  4. I think that scientist should have more preparation in human topics, because the fact that they are systematically missing the evidence that people way of learning is working very differently than the way they suppose, defies me.
    Being a scientist, I would take the evidence that my efforts are based on bad assumptions concerning the transfer of my message and work around it, and act in a different way as a consequence.

    If the conclusion is that we have to put the efforts, say, on a new religious movement, as suggested by John Michael Greer, so be it.

    It would be a paradoxical result, but also indeed a scientific and logic consequence.

    ReplyDelete
  5. The authors claim that "the immediate priority is to improve policy-makers' understanding of the imperfect nature of science. The essential skills are to be able to intelligently interrogate experts and advisers, and to understand the quality, limitations and biases of evidence."

    How can one argue with that?

    Nonetheless I would have preferred (or at least would have liked at least as much) authors who had claimed that "the (super) immediate priority is to improve scientists understanding of the (very) imperfect nature of policy-making. The essential skills are to be able to intelligently interrogate (policy) experts and advisers, and to understand the quality, limitations and biases of evidence (in policy-making)"

    That might have made scientists either relax a bit more or at least understand better what they are really up against.

    A third set of authors somewhere also would have done well to claim that "the immediate priority is to improve the public's understanding (of anything) including the imperfect nature of science and the imperfect nature of policy making. The essential skills are to be able to intelligently interrogate experts and advisers, and to understand the quality, limitations and biases of evidence." (in all fields). (including the field of understanding oneself)

    Once scientists, policy makers, and the public have all thoroughly understood the respective 20 key points to such improved and more enlightened understandings then perhaps a marginal bit of progress might start to be made on how to address the predicament we now find ourselves in.

    But until that happy moment is reached we will just have to continue to muddle through with incomplete understandings (if any at all), half truths, half measures, half effective communications and all the other half-assed things that somehow manage to very slowly become understood or done.

    Will the laws of physics take "a pause that refreshes"and wait for our massive and ubiquitous enlightenment?

    ReplyDelete