Written evidence submitted by Brigitte Nerlich (IPC0031)

How certain is certain? Conveying (un)certainty in the reporting of the 2013 5th IPCC report in English language media

Executive summary

 

  1. The Research Team

Professor Brigitte Nerlich and Dr Luke Collins submit evidence to the Select Committee of the House of Commons in response to the first publication of the Fifth IPCC Assessment Report. This work was conducted as part of the ESRC-funded project “From Greenhouse Effect to Climategate: A Systematic Study of Climate Change as a Complex Social Issue” [RES-360-25-0068]: a study examining climate change as a complex social issue and exploring the ways in which the debate around climate change impacts on social change and stability; the power and politics of namings, framings and metaphors; and the role of Web dynamics in public debate. The IPCC Assessment Report is seen as a work which aims to provide a broader understanding of the scientific view of climate change knowledge and its potential impact but the impact of the report is contingent upon the dissemination of its findings through global media. We examine the ways in which the media reports on the IPCC’s findings as a process that will determine how effective those findings can be.

  1. Introduction

Risks and uncertainties surrounding climate change are among the major issues on the scientific and political agenda. Scientists and the general public use the words risk and uncertainty in very different ways (Budescu et al., 2009; Campbell, 2011; Erkwurzel, et al., 2011; Harris et al., 2013; for more references and links see: http://blogs.plos.org/models/nine-lessons-and-carols-in-communicating-climate-uncertainty/). The notion of uncertainty is central to discussions of climate change, its causes, future trajectory and impacts, yet the term is open to many different interpretations and is represented in different ways by different groups (Leiserowitz et al., 2010; Newport, 2010).  It may be represented through statistical assessment, based on expert opinion, lay knowledge or gut instinct. Political action on climate change is hampered by scientists, policy makers, economists and the general public talking past each other, not understanding each other, or, in extreme cases, deliberately exploiting the inherent ambiguity of these terms to further their own objectives (http://www.nottingham.ac.uk/communicatingclimate/aboutus/project-team.aspx).

The House of Commons Select Committee asks: “How effective is AR5 and the summary for policymakers in conveying what is meant by uncertainty in scientific terms? Would a focus on risk rather than uncertainty be useful?”

In this submission we ask a slightly different (and slightly less ambiguous) question. Instead of focusing on policy makers understanding of the IPCC report, we want to investigate how effective the media were, when reporting on the IPCC report, in conveying what is meant by scientific certainty in lay people’s terms. We shift the focus of inquiry from policymakers understanding of the scientific foundations for assessments of uncertainty to public understanding of certainty. This understanding is mediated by newspapers and experts quoted in the newspapers. Studying how certainty is conveyed in news reports about the IPCC report will provide us with insights into the cultural context within which policymakers, who are of course also newspaper readers, may come to understand both certainty and uncertainty, both in scientific and in more commonsensical terms. We will see that this shift from the report itself to the reporting of the report entails a shift from (un)certainty to risk.

In 1999 The Science Technology Select Committee wrote the following in the midst of the BSE crisis:

When science and society cross swords, it is often over the question of risk. Risk, as is widely understood, has at least two dimensions: the chance of something happening, and the seriousness of the consequences if it does. It is often the case with new phenomena or theories that scientists are uncertain about both these things, and also uncertain about the chains of cause and effect supposedly at work. In this situation, any assurances which science may give must necessarily be hedged about with qualifications ("It appears to be safe, on the following assumptions which require further research"). Yet the public, or the media purporting to speak for the public, may demand unqualified assurances ("Is it 100 per cent safe?"), and may even perceive and present the response as being an unqualified assurance when it was not. By this means, the stage is set for confusion, cynicism and even panic.” (Science Technology Select Committee meeting, 1999: http://www.publications.parliament.uk/pa/ld199900/ldselect/ldsctech/38/3806.htm)

Fifteen years later and in the context of climate change, policy makers, scientists and the media face a similar situation but with a slightly different twist. The science of BSE or Bovine Spongiform Encephalopathy was relatively uncertain in around 1999 and still policy makers had to make decisions which affected the food chain, farmers and the health of the nation, that is, they had to make decisions under uncertainty. In the context of climate change the situation seems to be quite different.

Each successive IPCC report has claimed that scientific insights into the mechanisms and global impacts of climate change are more certain: that the global climate was warming and that the causes of this warming were anthropogenic in nature. There are still some uncertainties surrounding local impacts of climate change. We therefore have a situation in which policymakers are confronted with having to make decisions under conditions of relative certainty. To convey this certainty is a complex task fraught with difficulties, especially since other words are used too, such as confidence and consensus.

Climate scientists have developed various mechanisms to convey relative (un)certainties (and their confidence in asserting these) in scientific and lay people’s (including policy makers) terms which are carefully explained in the IPCC report.

However, both embers of the public and politicians tend to use the media as shortcuts to scientific information. In this article we therefore study how issues of (un)certainty are ‘conveyed’ or communicated in news reports on the IPCC report. This will provide us with some insights into the conceptual tools that publics and policy makers may use to think with when it comes to climate change.

Harris et al. say in a recent article: “Through the publication of probabilistic statements about climate risks and impacts, the IPCC has had a powerful influence on the concept of climate change in public and media discourses ... It is essential, therefore, to understand how the information that the IPCC conveys is understood by members of the public.” (2013: 415)

  1. Background

The Intergovernmental Panel on Climate Change (IPCC) is a intergovernmental body of scientists convened to assess the emerging science of climate change/global warming, review findings and make them available to policy makers. It was established in 1988. Its main activity is publishing special reports on climate change which have the agreement of leading climate scientists and the consensus of participating governments. The first report was published in 1988. Subsequent ones appeared in 1995, 2001, 2007 and 2013. A year before the third IPCC report, Moss and Schneider (2000) recommended a seven-step approach for describing uncertainty, including the use of specific qualitative language to conveying uncertainty in these reports so that the general public may understand them better. In 2003 Patt and Schrag (2003) commented on the specific language used to describe risk and probability in the Third Assessment Report and in 2005 Patt and Dessai (2005) commented on the ambiguity of the language of uncertainty.

The Guidance Note for Lead Authors of the IPCC Fourth Assessment Report on addressing Uncertainties (http://www.ipcc.ch/pdf/supporting-material/uncertainty-guidance-note_ar4.pdf) provides a detailed description of how uncertainty is treated in IPCC reports:

Where uncertainty is assessed qualitatively, it is characterised by providing a relative sense of the amount and quality of evidence (that is, information from theory, observations or models indicating whether a belief or proposition is true or valid) and the degree of agreement (that is, the level of concurrence in the literature on a particular finding). This approach is used by WG III through a series of self-explanatory terms such as: high agreement, much evidence; high agreement, medium evidence; medium agreement, medium evidence; etc.

Where uncertainty is assessed more quantitatively using expert judgement of the correctness of underlying data, models or analyses, then the following scale of confidence levels is used to express the assessed chance of a finding being correct: very high confidence at least 9 out of 10; high confidence about 8 out of 10; medium confidence about 5 out of 10; low confidence about 2 out of 10; and very low confidence less than 1 out of 10.

Where uncertainty in specific outcomes is assessed using expert judgment and statistical analysis of a body of evidence (e.g. observations or model results), then the following likelihood ranges are used to express the assessed probability of occurrence: virtually certain >99%; extremely likely >95%; very likely >90%; likely >66%; more likely than not > 50%; about as likely as not 33% to 66%; unlikely <33%; very unlikely <10%; extremely unlikely <5%; exceptionally unlikely <1%. (bold added).

Following the publication of the fourth IPCC assessment, Mastrandrea and Mach (2011) and Mastrandrea et al. (2011) published recommendations for the fifth assessment in terms of consistent treatment of uncertainties. Many of these authors were also involved in the Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties (November 2010). Their assessment called for a more consistent use of the calibrated language that had origins in the previous reports (see also Mastrandrea et al., 2010).

Within the report authors attempt to follow these now settled guidelines on how to represent uncertainty very closely, with varying degrees of success (see Harris et al., 2013). Previous research has also shown that even native speakers of English interpret the verbal probability expressions differently to the intention of the IPCC (see Budescu et al., 2009).

In the following we examine what happened when the fifth IPCC report and its uncertainty language is released into the wild so to speak. A first step of such a release is the press release. Astonishingly the press release of AR5 uses the expression ‘extremely likely’ without further definition when talking about human influence on global warming (http://www.ipcc.ch/news_and_events/docs/ar5/
press_release_ar5_wgi_en.pdf).

How were this press release and AR5 taken up and conveyed in the media? (A separate analysis is being undertaken of the press conference).

  1. Methods and materials

A search was conducted through LexisNexis for English-language publications using the terms ‘IPCC’ and ‘Intergovernmental Panel for Climate Change’. This search covered the timeframe between Friday 20th September 2013 and Saturday 5th October 2013 to accommodate the weekend news pieces published in the week preceding and the week following the release of the IPCC AR5 on Friday 27th September 2013.

A total of 2038 news items were collected across this 16-day period, including: newspaper articles (print and online); newswires; blogs; magazine articles; and transcribed television news broadcasts. These were collected from English-language news services in the U.K.; the U.S.; Canada; Australia and New Zealand; South Africa and continental African news services; as well as services in Asia such as United News of Bangladesh and The Bangkok Times.

Untitled.jpg

  1. Findings

The news items were analysed for reporting the deliberate use of qualitative and numerical terms to convey (un)certainty, as prescribed in the Guidance for Lead Authors of the IPCC AR5. Though many publications used ‘extremely likely’ and 95% certainty conterminously, only 59 news items alluded to the fact that such terms as ‘extremely likely’ were used in the report as part of a calibrated language scale in relation to levels of (un)certainty. Of those 59 news items, 55 made only a passing reference to the IPCC definitions of “extremely likely” and “virtually certain”.

Throughout the timeframe, reference to the report’s specific terms appears as ‘definition’ with no further explanation of alternative terms or probabilities. Some examples:

“they will probably release a report saying it is "extremely likely" which they define in footnotes as 95 percent certain that humans are mostly to blame for temperatures that have climbed since 1951.”

“In the IPCC's lexicon, "extremely likely" equates to 95 per cent certainty.”

"This report uses the term 'virtually certain', which is defined as 99-100% certainty, that human activity is causing global warming.”

5.1 Elaboration is minimal

Only four news items, mainly blogs, engaged with the issue of language itself. Two examples:

“Yet if the recent past is an indication of the future, deniers will fixate on the word "likely", because "likely" in the language of the layman doesn't mean "certain". Scientists don't speak in the language of certainty: to them, 95% certainty means "you can take it to the bank".” (The Huffington Post Web Blog U.S. Ignoring Paul Revere 28/09/2013).

“Stott said, 'You can then look back over the past assessments, and you can see how the level of confidence has increased... I would just like to go back and say very basically why we have come to that high level of confidence.' The fact that the methodology used is carefully calibrated across all the reports, says Stott, means that the results from each are directly comparable. 'We have very importantly and very carefully calibrated that system [used in AR5] against previous assessments. 'We deliberately took the lead in the fourth assessment report to do the same type of analysis and of course use the same type of calibrated language, use the same kind of robust and rigorous approach, in order to compare our assessments.” (Responding to Climate Change Web Blog U.K. IPCC's growing confidence is 'not a question of grade inflation' 04/10/2013).

5.2 ‘95% certainty’ is translated through analogies

The most quoted ‘statistic’ in the media reporting on the firth IPCC report is that the IPCC has increased it’s certainty level from 90% in the last report to 95% in this report. Some news items attempt to make this statistical pronouncement more transparent or familiar through the use analogies, mostly taken from assessing risks around smoking, cancer, surgery or flu diagnoses, insurance policies and the risk of houses catching fire, car break-downs, flying and so on. Some comparisons are also made with various widely known scientific phenomena such as gravity or, perhaps less widely known ones, such as the age of the universe. Tony Blair was the only commentator who based his analogy on Elvis Presley. Here are some examples:

-          "Ninety-five per cent certain, is a pretty large degree of certainty - I recall that the number of people who think Elvis is still alive is round about five per cent," he [Tony Blair] added, to laughter.

-          But Sherwood strongly rejects suggestions the changes are an admission of past errors. He returns to his cancer patient analogy: if the diagnosis is the patient is going to die, but there is some uncertainty on exactly when, you do not just throw out the initial diagnosis of death.

-          Nobody likes bad news, but if 97 mechanics say my car is overheating because it has been mistreated and three others -- plus the butcher, the baker and the candlestick maker -- say it's just a glitch and it'll fix itself, who should I listen to?

-          Top scientists from a variety of fields say they are about as certain that global warming is a real, man-made threat as they are that cigarettes kill.

-          They said gravity is a good example of something more certain than climate change. Climate change "is not as sure as if you drop a stone it will hit the Earth," said Princeton University climate scientist Michael Oppenheimer. "It's not certain, but it's close."

-          Arizona State University physicist Lawrence Krauss said the 95 per cent quoted for climate change is equivalent to the current certainty among physicists that the universe is 13.8 billion years old.

-          "What is understood does not violate any mechanism that we understand about cancer," while "statistics confirm what we know about cancer," said Cicerone, an atmospheric scientist. Add to that a "very high consensus" among scientists about the harm of tobacco, and it sounds similar to the case for climate change, he said.

-          "If you were 95 per cent certain your house was at risk of catching fire, and if there was something you could do to prevent it, you would do it," Bruce said.

-          "if someone came to you and said there is a 95 per cent chance that your house might burn down, even if you are in the 5 per cent that doesn't agree with it, you still take out the insurance, just in case."

-          The EU's commissioner for climate action, Connie Hedegaard, said: "What would you do if your doctor was 95 per cent sure you had a serious illness? Whose side are you on? Those who want to act on 95 per cent certainty or those who gamble on the remaining 5 per cent?"

However, there were some news items, mainly letters to the editor, in which people said that 95% certainty was not enough, they still wanted 100%.

-          Of all the unlikely IPCC statements on human-induced global warming, its announcement of "95 per cent certainty" that this is the case fits in with previous dodgy statistics. Whether it identifies 97 per cent or 83 per cent "certainty", it simply means that it is not certain.

-          SIR - So the IPCC is 95 per cent certain that mankind has been the main cause of climate change. Do they simply mean that 19 out of every 20 of their scientists agree?

The analogies for ‘95% certainty’ evoke different levels of risk – from fatal illness to getting caught in the rain. Such comparisons reiterate the association of uncertainty and risk and that numbers alone are not universal. We have to consider the risk in relation to consequences, whereby the severity and magnitude of those consequences eschews the level of uncertainty we might be willing to accept.

  1. Conclusion

Conveying scientific information, including information about levels of (un)certainty, is difficult. It is what one may call a Goldilocks problem (other experts in science communication call it the 'babel fish dilemma', http://blog.wellcome.ac.uk/2012/10/29/lost-in-translation/). Too much purely scientific information may put readers off reading further, whereas a rashly chosen analogy may provide readers with a potentially false feeling of understanding. It is therefore necessary in some instances to "point out the analogy's limits to the reader in order to avoid incorrect extrapolations" (http://www.scidev.net/global/communication/practical-guide/-the-little-manual-on-science-communication-a-summ.html). Calibrating the language of climate science is only a first step in conveying climate science to various publics, including policy makers.

 

December 2013

  1. References

Budescu D.V., Broomell, S. and Por, H. (2009). “Improving communication of uncertainty in the reports of the intergovernmental panel on climate change.” Psychological Science 20(3) 299-308. doi: 10.1111/j.1467-9280.2009.02284.x.

 

Budescu D.V., Por, H.-H. and Broomell, S. B. (2012) “Effective communication of uncertainty in the IPCC reports.” Climatic Change 113(2) 181-200.

 

Campbell, P. (2011) “Understanding the receivers and the reception of science’s uncertain messages.” Philosophical Transactions of the Royal Society A 369(1956) 4891-4912. doi: 10.1098/rsta.2011.0068.

 

Edwards, T. (6 December, 2013) “Nine Lessons and Carols in Communicating Climate Uncertainty.” PLOS Blogs. Available at: http://blogs.plos.org/models/nine-lessons-and-carols-in-communicating-climate-uncertainty. Accessed 9th December, 2013.

 

Erkwurzel, B., Frumhoff, P. C. and McCarthy, J. J. (2011) “Climate uncertainties and their discontents: increasing the impact of assessments on public understanding of climate risks and choices.” Climate Change 108(4) 791-802.

 

Harris, A. J. L., Corner, A., Xu, J. and Du, X. (2013) “Lost in translation? Interpretations of the probability phrases used by the Intergovernmental Panel on Climate Change in China and the UK.” Climatic Change 121 415-425. doi: 10.1007/s10584-013-0975-1.

 

Leiserowitz, A., Smith, N. & Marlon, J.R. (2010) “Americans’ Knowledge of Climate Change.” Yale

University. New Haven, CT: Yale Project on Climate Change Communication.

http://environment.yale.edu/climate/files/ClimateChangeKnowledge2010.pdf.

 

Mastrandrea, M.D., Field, C.B., Stocker, T.F., Edenhofer, O., Ebi, K.L., Frame, D.J., Held, H., Kriegler, E., Mach, K.J., Matschoss, P.R., Plattner, G.-K., Yohe, G.W. and Zwiers, F.W. (2010) “Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties.” Intergovernmental Panel on Climate Change (IPCC). Available at <http://www.ipcc.ch>.

 

Mastrandrea, M. D. and Mach, K. J. (2011) “Treatment of uncertainties in IPCC Assessment Reports: past approaches and considerations for the Fifth Assessment Report.” Climatic Change 108 659-673. doi: 10.1007/s10584-011-0177-7.

 

Mastrandrea, M. D., Mach, K. J., Plattner, G.-P., Edenhofer, O., Stocker, T. F., Field, C. B., Ebi, K. L. and Matschoss, P. R. (2011) “The IPCC AR5 guidance note on consistent treatment of uncertainties: a common approach across the working groups.” Climatic Change 108(4) 675-691.

 

Moss, R. H. and Schneider, S. H. (2000). Uncertainties in Guidance papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC. Edited by R. Pachauri, T. Taniguchi and K. Tanaka. Intergovernmental Panel on Climate Change (IPCC), Geneva. Available at: http://www.ipcc.ch/pdf/supporting-material/guidance-papers-3rd-assessment.pdf.

 

Newport, F. (11th March, 2010) “Americans’ Global Warming Concerns Continue To Drop”. GALLUP Politics. Available at: www.gallup.com/poll/126560/americans-global-warming-concerns-continue-drop.aspx. Accessed 9th December 2013.

 

Patt, A. and Schrag, D. P. (2003) “Using specific language to describe risk and probability.” Climate Change 61 17-30.

 

Patt, A. and Dessai, S. (2005) “Communicating uncertainty: lessons learned and suggestions for climate change assessment.” Comptes Rendus Geoscience 337(4) 425-441.

 

Roche, R. (29 October, 2012) “Lost in translation: the dangers of using analogies in science.” Wellcome Trust. Available at: http://blog.wellcome.ac.uk/2012/10/29/lost-in-translation/. Accessed 9th December 2013.