Thursday, October 21, 2021

Have Past IPCC Temperature Projections/Predictions Been Accurate?

Arguments for or against doing things to slow climate change depend on what will happen if we don’t, a question the IPCC reports try to answer. That makes it important to know how reliable their predictions are. The latest report runs to almost four thousand pages, largely of detailed analysis depending on multiple scholarly articles for each step — Chapter 7, to pick one at random, has fifty editors and about nine hundred articles in its list of references. Someone with infinite time, energy and expertise might be able to go through all of the calculations that produced the predictions in the reports in order to see if they were done correctly, but that is not a practical option.

There is an alternative. The climate system is too complicated to make predictions on the basis of theory alone, hence the IPCC project largely consists of sophisticated curve fitting, picking a form for the relationship among observables suggested by physical theory, choosing parameters for the relationships, how strong each effect is, by finding the values that best fit historical data.  With enough tweaking of the models and adjusting of parameters that process can fit past data, but that does not tell you whether the models fit the real system well enough to correctly predict future data. As someone is supposed to have said, with enough parameters you can fit the skyline of New York.

The solution, for both the researcher who wants to know if his model is right and someone else trying to decide whether to believe him, is to test the model against data that were not used in creating it. We do not know the future, the future eventually becomes the past, so a model constructed in 1990 can be tested in 2021 against data that did not exist when the model was constructed.

The past reports are webbed. Back in 2014 I looked at each to see what someone who read it would expect future temperature to do and reported the results on my blog. If you would like to check my conclusions about what each report implied for yourself you can find links to the reports here.

What the IPCC Predicted

The executive summary of the first report (1990) contains:

Under the IPCC Business-as-Usual (Scenario A) emissions of greenhouse gases, the average rate of increase of global mean temperature during the next century is estimated to be about 0.3°C per decade (with an uncertainty range of 0.2°C to 0.5°C).

The graph shown for the increase is close to a straight line at least from 2000 on, so it seems reasonable to ask whether the average increase from 1990 to the present is within that range. 

Figure 18 from the Second Assessment Report (1995) shows the future temperature through 2020. Through that date, it rises steadily at about .14°C/decade.[1]

From the Third Assessment Report (2001):[2]

For the periods 1990 to 2025 and 1990 to 2050, the projected increases are 0.4 to 1.1°C and 0.8 to 2.6°C, respectively.

For the former period, that implies an increase of from .11 to .31 °C/decade.

The Fourth Assessment Report (2007) has[3]

For the next two decades a warming of about 0.2°C per decade is projected for a range of SRES emissions scenarios.

What Happened

When I did the calculations in 2014, I found that the IPCC had predicted high four times out of four, twice by enough so that actual warming was below the bottom of the predicted range. That looked like evidence that we should not put much weight on their predictions of future temperature.

We now have seven years more data, so I did it again. As of September of 2021, when I am writing this, the last year whose temperature is shown on the NASA page I am using is 2018; I have redone the calculations accordingly. Here are the results:

The first IPCC report was released in 1990. From then to 2018, global temperature rose .38°C for an average of .14°C/decade, well below the predicted range.

The second report was released in 1995. From then to 2018, temperature rose by .37°C, for an average rate of growth of .16 °C, a little higher than the prediction.


The third report was released in 2001. From then to 2018, temperature rose by .29°C for an average of .17°C/decade, towards the lower end of the predicted range.


The fourth report was released in 2007. From then until 2018, temperature rose by .18 degrees, .16°C/decade, below the predicted .2°C.


The predictions look better now than they did in 2014, high three times out of four, low once, and only once has actual warming been below the predicted range. They are still running a little high but the results look consistent with random error. That makes it at least possible that the IPCC researchers are now modeling the climate system well enough to produce reasonable estimates of its future behavior.

It is possible but far from certain because the test they passed is not a very strong one. A theory that correctly predicted the outcome of next year’s elections, including every house seat, every senate seat, and the total votes for each party, would be a very good theory indeed, since doing that well by chance is very unlikely, so we would have good reason to trust its future predictions. A theory which correctly predicted which party will end up with a senate majority after the 2022 election would be better than one that got it wrong but not much better, since one can get the right answer half the time by flipping a coin.

The IPCC reports rely on complicated models and a lot of data. One way to judge how impressive their results are, how much evidence that they have done a good job of modelling climate, is to compare their results with those of much simpler models. The simplest is the assumption that global temperature never changes. The IPCC did a little worse than that model in 1990, since it predicted warming from then to 2018 of .3°/decade and the actual value,  .14°/decade, was closer to zero. But they did much better than that model the next three times.

The next simplest model is a straight line. From 1910, about when current warming started, to 1990, when the first IPCC report came out, warming was .11 °C/decade. The rate of warming from 1990 to 2018 was .14 °C/decade, so the straight-line prediction made in 1990 predicts about 79% of warming from then to 2018. The ratio of the IPCC prediction to what actually happened was 250% for the 1990 prediction, 81% for the 1995, 125% for the 2001 and 2007 predictions.[4] So predicting that the rate of warming would continue at its average level as of 1990 does much better than the 1990 prediction, about as well as the three later ones.

One test of how good the IPCC models are is to see how well each of them did at predicting warming from then to now. The first report fails that test, the next three pass it; actual warming was within their predicted range although not equal to their best guess. That is evidence that those models can be expect to give correct predictions in the future but not very strong evidence. 

It is, however, much better than the evidence was in 2014.

[1] The figure is on page 323 of Climate Change 1995 The Science of Climate Change. When I did my calculations in 2014 I thought it was .13°/decade but measuring the graph more carefully I now think it is .14.

[4] I am defining each of the IPCC predictions as the predicted value if there is one or the center of the range if there isn’t.


A Country Framer said...

If their predictions might be getting more accurate if yet a bit too warm, are you worried about the implications? In other words, if they are roughly correct, does that worry you (assuming current trends of economic growth)? Or is there an extra premise needed of some sort of "escape velocity" of temperature growth that would lead to existential crisis?

David Friedman said...

It doesn't much worry me. Their predictions for the end of the century involve considerably less change in temperature than the current range of climates that people live and function over — by the end of the century Minnesota may be as warm as Iowa is now. I think the future is sufficiently uncertain, due mostly to technological progress, that taking actions now on the basis of what we think will happen more than a century out is almost always a mistake.

As I have argued at length on this blog, climate change has positive and negative effects uncertain in size and there is no good reason to expect the net effect to be negative, although it could be. There is no reason I can see to expect the net to be not only negative but catastrophic, a position many people hold although it is not supported by the most recent IPCC report.

Gary Y. said...

I'm somehow reminded of the old story, "The boy who cried Wolf."

Unknown said...

Climate models aren't exercises in curve-fitting. For example, the presence of ENSO is an emergent phenomena in GCMs - they're not tuned to create ENSO specifically.

The change in the mean creates a change in the extremes - and that's where the pain hits. A small change in the mean can change the frequency/intensity of a mean by a far greater amount. Hence the off-the-charts Pacific NW heat wave this summer. Record highs weren't just barely passed, they were obliterated.

You're also ignoring shifts in circulation patterns and in precipitation - anthropogenic climate change isn't just warming temperatures. Minnesota may only get as warm as Iowa, but if its precipitation becomes more like New Mexico, that's a bigger problem.

There is no good reason to expect the net effect to be positive, either. Erring on the side of optimism comes off as naive.

SB said...

The projections you quote from the 1990 report are "business-as-usual emissions scenario"; if the world actually has taken some emission-reduction actions over the past thirty years, that would explain why those projections are more dramatic than actually happened. Are the projections you quote from subsequent reports likewise based on "business-as-usual emissions scenarios"?

On the question of positive and negative impacts... it's certainly possible that a planet a degree or two warmer would be just as habitable as the one we grew up with, albeit with more people living in Canada and Siberia and fewer in the tropics. But it's the rapid transition that worries me: a degree-or-two change over the course of a century will be a substantial geopolitical and economic challenge, forcing many millions of (mostly-poor) people to migrate, all in the same direction at the same time, and a much greater challenge for non-human species, many of which will go extinct because this is a faster change than they can adapt to.

David Friedman said...

A sufficiently rapid change would be a problem, but a degree or two in a century is not rapid. Take a look at a list of average temperatures by U.S. state to see how small a difference that is. It doesn't require mass migration.

Note also that, as per the latest IPCC report, warming is lower in hot areas. Warming in India is projected as lower than the average, in the arctic as much higher.

Sea level rise of a meter or so doesn't require mass migration either — that's about half the difference between high tide and low.

Anonymous said...

A global mean temperature increase of 2 to 3 degrees C (which is where we’re headed; we’ve already baked in 1 degree C) in a century so is incredibly rapid, David.

You’re still missing the changes in extremes caused by a shift in the mean. Why is that?

A meter of sea level rise beyond the highest tide is a big deal. Sandy wouldn’t have flooded the NYC subways if not for the sea level rise already in place.

David Friedman said...


Which changes in extremes are you describing? Table 11.SM.2 of the sixth report shows the projected effect on the hottest day of various levels of warming. Typically it goes up by between a degree and a degree and a half for every degree of warming. Figure 11.SM.1 shows the effect on minimum temperature, with the temperature of the coldest day typically going up by two to three degrees for each degree of warming, although less in (typically) warmer climates. Given that both heat and cold kill, with cold apparently killing considerably more than heat, that looks like a gain for humans.

So far as Sandy, are you saying that eight inches of sea level rise (A.1.7 gives .2 meters from 1902 to 2018) were responsible for flooding subways? I thought the problem was that it hit at high tide, which is about a six foot difference.

By what standard is 2° in a century incredibly rapid? That's equivalent to moving from Iowa to Indiana.

Anonymous said...

Consider the record-obliterating heat waves that have taken place around the globe for the last 10-20 years. Those have occurred with only ~1ºC warming, and haven't been only about 1 1/2ºC warmer than previous high extremes. The old trope about cold killing more than heat was never really proven and by now is a sad joke.

NYC Battery Park has observed nearly a foot of SLR rise since pre-industrial times. That foot was the critical difference - and noting that the magnitude is smaller is irrelevant. Find the videos of the 11'8" bridge - a two-inch lack of clearance opens a panel truck like a sardine can.

What other events have caused a global average temperature rise of the same magnitude that was permanent?

Unknown said...

David, thanks for your effort, in particular, the zero and trend following alternatives. In terms of falsification, there would be even more to verify: 1. alrenative prediction questions to establish the causality: e.g. why CO2 and not plain land usage or population growth? 2. The fear-inducing impact forecasts. We can always enter a bunch of long-term impact predictions for automatic verification. Cheers, Hubertus

Hubertus said...

David, thanks for your effort, in particular, the zero and trend following alternatives. In terms of falsification, there would be even more to verify: 1. alternative prediction questions which establish the true causality: e.g. why CO2 and not plain land usage or population growth? 2. Verifications of fear-inducing impact forecasts. We can always enter a bunch of long-term impact predictions for automatic reality checks. Cheers, Hubertus