Abstract
Climate model ensembles are used to estimate uncertainty in future projections, typically by interpreting the ensemble distribution for a particular variable probabilistically. There are, however, different ways to produce climate model ensembles that yield different results, and therefore different probabilities for a future change in a variable. Perhaps equally importantly, there are different approaches to interpreting the ensemble distribution that lead to different conclusions. Here we use a reduced-resolution climate system model to compare three common ways to generate ensembles: initial conditions perturbation, physical parameter perturbation, and structural changes. Despite these three approaches conceptually representing very different categories of uncertainty within a modelling system, when comparing simulations to observations of surface air temperature they can be very difficult to separate. Using the twentieth century CMIP5 ensemble for comparison, we show that initial conditions ensembles, in theory representing internal variability, significantly underestimate observed variance. Structural ensembles, perhaps less surprisingly, exhibit over-dispersion in simulated variance. We argue that future climate model ensembles may need to include parameter or structural perturbation members in addition to perturbed initial conditions members to ensure that they sample uncertainty due to internal variability more completely. We note that where ensembles are over- or under-dispersive, such as for the CMIP5 ensemble, estimates of uncertainty need to be treated with care.
Similar content being viewed by others
References
Annan JD, Hargreaves JC (2010) Reliability of the CMIP3 ensemble. Geophys Res Lett 37:5. doi:10.1029/2009GL041994. http://www.agu.org/pubs/crossref/2010/2009GL041994.shtml
Bishop CH, Abramowitz G (2013) Climate model dependence and the replicate earth paradigm. Clim Dyn 41(3-4):885–900. doi:10.1007/s00382-012-1610-y
Brohan P, Kennedy JJ, Harris I, Tett SFB, Jones PD (2006) Uncertainty estimates in regional and global observed temperature changes: a new data set from 1850. J Geophys Res 111(D12). doi:10.1029/2005JD006548. http://www.agu.org/pubs/crossref/2006/2005JD006548.shtml
Ducharne A, Laval K (2000) Influence of the realistic description of soil water-holding capacity on the global water cycle in a GCM. J Clim 13(24):4393–4413. doi:10.1175/1520-0442(2000)013<4393:IOTRDO>2.0.CO;2. http://journals.ametsoc.org/doi/abs/10.1175/1520-0442(2000)013<4393%3AIOTRDO>2.0.CO%3B2
Fernández-Donado L, González-Rouco JF, Raible CC, Ammann CM, Barriopedro D, García-Bustamante E, Jungclaus JH, Lorenz SJ, Luterbacher J, Phipps SJ, Servonnat J, Swingedouw D, Tett SFB, Wagner S, Yiou P, Zorita E (2013) Large-scale temperature response to external forcing in simulations and reconstructions of the last millennium. Clim Past 9(1):393–421. doi:10.5194/cp-9-393-2013. http://www.clim-past.net/9/393/2013/
Fischer EM, Lawrence DM, Sanderson BM (2010) Quantifying uncertainties in projections of extremesa perturbed land surface parameter experiment. Clim Dyn 37(7-8):1381–1398. doi:10.1007/s00382-010-0915-y,
Hamill TM (2001) Interpretation of rank histograms for verifying ensemble forecasts. Mon Weather Rev 129(3):550–560. doi:10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2. http://journals.ametsoc.org/doi/abs/10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO%3B2
Knutti R, Abramowitz G, Collins M, Eyering V, Gleckler PJ, Hewitson B, Mearns LO (2010) Good practice guidance paper on assessing and combining multi model climate projections. In: Stocker TF, Qin D, Plattner GK, Tignor M, Midgley GF (eds) Meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. IPCC Working Group I Technical Support Unit. University of Bern, Bern, Switzerland
Macadam I, Pitman AJ, Whetton PH, Abramowitz G (2010) Ranking climate models by performance using actual values and anomalies: implications for climate change impact assessments. Geophys Res Lett 37(16). doi:10.1029/2010GL043877. http://www.agu.org/pubs/crossref/2010/2010GL043877.shtml
Meehl GA, Stocker TF, Collins WD, Friedlingstein P, Gaye AT, Gregory JM, Kitoh A, Knutti R, Murphy JM, Noda A, Raper SC, Watterson IG, Weaver AJ, Zhao ZC (2007) Global climate projections. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds) Climate change 2007: the physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, pp 747–845
Murphy J, Sexton D, Barnett D, Jones G, Webb M, Collins M, Stainforth D (2004) Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature 430(7001):768–772. http://www.nature.com/nature/journal/v430/n7001/abs/nature02771.html
Oldenborgh GJv, Reyes FJD, Drijfhout SS, Hawkins E (2013) Reliability of regional climate model trends. Environ Res Lett 8:014-055. doi:10.1088/1748-9326/8/1/014055. http://iopscience.iop.org/1748-9326/8/1/014055
Perkins SE, Pitman AJ, Holbrook NJ, McAneney J (2007) Evaluation of the AR4 climate models simulated daily maximum temperature, minimum temperature, and precipitation over australia using probability density functions. J Clim 20(17):4356–4376. doi:10.1175/JCLI4253.1
Phipps SJ (2010) The CSIRO Mk3L climate system model v1.2, Technical report no. 4. The Antarctic Climate & Ecosystems CRC, Hobart, Tasmania, Australia
Phipps SJ, Rotstayn LD, Gordon HB, Roberts JL, Hirst AC, Budd WF (2011) The CSIRO Mk3L climate system model version 1.0—part 1: description and evaluation. Geosci Model Dev 4(2):483–509. doi:10.5194/gmd-4-483-2011. http://www.geosci-model-dev.net/4/483/2011/
Phipps SJ, Rotstayn LD, Gordon HB, Roberts JL, Hirst AC, Budd WF (2012) The CSIRO Mk3L climate system model version 1.0—part 2: response to external forcings. Geosci Model Dev 5(3):649–682. doi:10.5194/gmd-5-649-2012. http://www.geosci-model-dev.net/5/649/2012/
Phipps SJ, McGregor HV, Gergis J, Gallant AJE, Neukom R, Stevenson S, Ackerley D, Brown JR, Fischer MJ, van Ommen TD (2013) Paleoclimate data-model comparison and the role of climate forcings over the past 1500 years. J Clim doi:10.1175/JCLI-D-12-00108.1
Pirani A (ed) (2008) WCRP Coupled Model Intercomparison Project—Phase 5, CLIVAR echanges, vol 16. National Oceanography Centre, Southampton, UK. http://cedadocs.badc.rl.ac.uk/310/1/feb2008.pdf
Randall D, Wood R, Bony S, Colman R, Fichefet T, Fyfe J, Kattsov V, Pitman A, Shukla J, Srinivasan J, Stouffer R, Sumi A, Taylor K (2007) Climate models and their evaluation. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt K, Tignor M, Miller H (eds) Climate change 2007: the physical science basis. Contribution of working group I to the fourth assessment report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge
Reichert P, Schervish M, Small MJ (2002) An efficient sampling technique for bayesian inference with computationally demanding models. Technometrics 44(4):318–327. doi:10.1198/004017002188618518
Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt K, Tignor M (eds) (2007) Contribution of working group I to the fourth assessment report of the Intergovernmental Panel on Climate Change. IPCC Fourth Assessment Report: climate change 2007. Cambridge University Press, Cambridge. http://www.ipcc.ch/publications_and_data/publications_ipcc_fourth_assessment_report_wg1_report_the_physical_science_basis.htm
Stainforth D, Aina T, Christensen C, Collins M, Faull N, Frame D, Kettleborough J, Knight S, Martin A, Murphy J, et al (2005) Uncertainty in predictions of the climate response to rising levels of greenhouse gases. Nature 433(7024):403–406. doi:10.1038/nature03301. http://www.nature.com/nature/journal/v433/n7024/abs/nature03301.html
Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philos T Roy Soc A 365(1857):2053–2075
Washington WM, Meehl GA (1989) Climate sensitivity due to increased CO2 experiments with a coupled atmosphere and ocean general circulation model. Clim Dyn 4(1):1–38. doi:10.1007/BF00207397. http://www.springerlink.com/content/k13643612510589x/abstract/
WCRP (2013) CMIP5: overview. http://cmip.llnl.gov/cmip5/index.html
Yokohata T, Annan JD, Collins M, Jackson CS, Tobis M, Webb MJ, Hargreaves JC (2012) Reliability of multi-model and structurally different single-model ensembles. Clim Dyn pp 1–18. doi:10.1007/s00382-011-1203-1. http://www.springerlink.com/index/Y51442386248051N.pdf
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Haughton, N., Abramowitz, G., Pitman, A. et al. On the generation of climate model ensembles. Clim Dyn 43, 2297–2308 (2014). https://doi.org/10.1007/s00382-014-2054-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-014-2054-3