Abstract
Projections based on climate model ensembles commonly assume that each individual model simulation is of equal value. When combining simulations to estimate the mean and variance of quantities of interest, they are typically unweighted. Exceptions to this approach usually fall into two categories. First, ensembles may be pared down by removing either poorly performing model simulations or model simulations that are perceived to add little additional information, typically where multiple simulations have come from the same model. Second, weighting methodologies, usually based on model performance differences, may be applied, and lead to some improvement in the projected mean. Here we compare the effect of three different weighting techniques—simple averaging, performance based weighting, and weighting that accounts for model dependence—on three ensembles generated by different approaches to model perturbation. We examine the effect of each weighting technique on both the ensemble mean and variance. For comparison, we also consider the effect on the CMIP5 ensemble. While performance weighting is shown to improve the estimate of the mean, it does not appear to improve estimates of ensemble variance, and may in fact degrade them. In contrast, the model independence weighting approach appears to improve both the ensemble mean and the variance in all ensembles.
Similar content being viewed by others
References
Abramowitz G, Bishop CH (2014) Climate model dependence and the ensemble dependence transformation of CMIP projections. J Clim. doi:10.1175/JCLI-D-14-00364.1. http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-14-00364.1
Annan JD (2010) IPCC experts new clothes. http://julesandjames.blogspot.com.au/2010/08/ipcc-experts-new-clothes.html
Annan JD, Hargreaves JC (2010) Reliability of the CMIP3 ensemble. Geophys Res Lett 37:5. doi:10.1029/2009GL041994. http://www.agu.org/pubs/crossref/2010/2009GL041994.shtml
Bishop CH, Abramowitz G (2013) Climate model dependence and the replicate earth paradigm. Clim Dyn 41(3–4):885–900. doi:10.1007/s00382-012-1610-y
Brohan P, Kennedy JJ, Harris I, Tett SFB, Jones PD (2006) Uncertainty estimates in regional and global observed temperature changes: a new data set from 1850. J Geophys Res 111(D12). doi:10.1029/2005JD006548. http://www.agu.org/pubs/crossref/2006/2005JD006548.shtml
Flato G, Marotzke J, Abiodun B, Braconnot P, Chou SC, Collins W, Cox P, Driouech F, Emori S, Eyring V, Forest C, Gleckler P, Guilyardi E, Jakob C, Kattsov V, Reason C, Rummukaine M (2013) Evaluation of climate models. In: Stocker T, Qin D, Plattner GK, Tignor M, Allen S, Boschung J, Nauels A, Xia Y, Bex V, Midgley P (eds) Climate change 2013: the physical science basis. Contribution of working group I to the fifth assessment report of the intergovernmental panel on climate change, Cambridge University Press, Cambridge. http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
Giorgi F (2005) Climate change prediction. Clim Change 73(3):239–265. doi:10.1007/s10584-005-6857-4
Gleckler P, Taylor K, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113(D6):D06,104. doi:10.1029/2007JD008972. http://www.agu.org/pubs/crossref/2008/2007JD008972.shtml
Haughton N, Abramowitz G, Pitman AJ, Phipps SJ (2014) On the generation of climate model ensembles. Clim Dyn, pp 1–12. doi:10.1007/s00382-014-2054-3
Hegerl GC, Zwiers FW, Braconnot P, Gillett NP, Lou Y, Marengo Orsini JA, Nicholls N, Penner JE, Stott PA (2007) Understanding and attributing climate change. In: Understanding and attributing climate change, Cambridge University Press, Cambridge. http://www.ipcc.ch/publications_and_data/publications_ipcc_fourth_assessment_report_wg1_report_the_physical_science_basis.htm
Knutti R, Abramowitz G, Collins M, Eyering V, Gleckler PJ, Hewitson B, Mearns LO (2010a) Good practice guidance paper on assessing and combining multi model climate projections, IPCC working group I technical support unit. In: Stocker TF, Qin D, Plattner GK, Tignor M, Midgley GF (eds) Meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. University of Bern, Bern
Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010b) Challenges in combining projections from multiple climate models. J Clim 23(10):2739–2758. doi:10.1175/2009JCLI3361.1. http://0-journals.ametsoc.org.library.newcastle.edu.au/doi/abs/10.1175/2009JCLI3361.1
Krishnamurti TN, Kishtawal CM, Zhang Z, LaRow T, Bachiochi D, Williford E, Gadgil S, Surendran S (2000) Multimodel ensemble forecasts for weather and seasonal climate. J Clim 13(23):4196–4216. doi:10.1175/1520-0442(2000)013. http://journals.ametsoc.org/doi/abs/10.1175/1520-0442(2000)013%3C4196:MEFFWA%3E2.0.CO%3B2
Macadam I, Pitman AJ, Whetton PH, Abramowitz G (2010) Ranking climate models by performance using actual values and anomalies: implications for climate change impact assessments. Geophys Res Lett 37(16). doi:10.1029/2010GL043877. http://www.agu.org/pubs/crossref/2010/2010GL043877.shtml
Masson D, Knutti R (2011) Climate model genealogy. Geophys Res Lett 38(8). doi:10.1029/2011GL046864
PAGES 2k Consortium (2013) Continental-scale temperature variability during the past two millennia. Nature Geosci 6(5):339–346. doi:10.1038/ngeo1797. http://www.nature.com/ngeo/journal/vaop/ncurrent/full/ngeo1797.html
Phipps SJ, Rotstayn LD, Gordon HB, Roberts JL, Hirst AC, Budd WF (2011) The CSIRO Mk3L climate system model version 1.0–part 1: description and evaluation. Geosci Model Dev 4(2):483–509. doi:10.5194/gmd-4-483-2011. http://www.geosci-model-dev.net/4/483/2011/
Phipps SJ, Rotstayn LD, Gordon HB, Roberts JL, Hirst AC, Budd WF (2012) The CSIRO Mk3L climate system model version 1.0–part 2: response to external forcings. Geosci Model Dev 5(3):649–682. doi:10.5194/gmd-5-649-2012. http://www.geosci-model-dev.net/5/649/2012/
Phipps SJ, McGregor HV, Gergis J, Gallant AJE, Neukom R, Stevenson S, Ackerley D, Brown JR, Fischer MJ, van Ommen TD (2013) Paleoclimate data–model comparison and the role of climate forcings over the past 1500 years. J Clim 26(18):6915–6936. doi:10.1175/JCLI-D-12-00108.1. http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-12-00108.1
Potempski S, Galmarini S (2009) Est modus in rebus: analytical properties of multi-model ensembles. Atmos Chem Phys 9(24):9471–9489. doi:10.5194/acp-9-9471-2009. http://www.atmos-chem-phys.net/9/9471/2009/
Randall D, Wood R, Bony S, Colman R, Fichefet T, Fyfe J, Kattsov V, Pitman AJ, Shukla J, Srinivasan J, Stouffer R, Sumi A, Taylor K (2007) Climate Models and their Evaluation. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt K, Tignor M, Miller H (eds) Cilmate models and their evaluation. In: Climate change 2007: the physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change, Cambridge University Press, Cambridge
Reichert P, Schervish M, Small MJ (2002) An efficient sampling technique for bayesian inference with computationally demanding models. Technometrics 44(4):318–327. doi:10.1198/004017002188618518. http://www.tandfonline.com/doi/abs/10.1198/004017002188618518
Reifen C, Toumi R (2009) Climate projections: past performance no guarantee of future skill? Geophys Res Lett 36(13). doi:10.1029/2009GL038082. http://www.agu.org/pubs/crossref/2009/2009GL038082.shtml
Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt K, Tignor M (eds) (2007) Contribution of working group i to the fourth assessment report of the intergovernmental panel on climate change. IPCC fourth assessment report: climate change 2007. Cambridge University Press, Cambridge. http://www.ipcc.ch/publications_and_data/publications_ipcc_fourth_assessment_report_wg1_report_the_physical_science_basis.htm
Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93(4):485–498. doi:10.1175/BAMS-D-11-00094.1. http://journals.ametsoc.org/doi/abs/10.1175/BAMS-D-11-00094.1
Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philos T Roy Soc A 365(1857):2053–2075
Weigel AP, Knutti R, Liniger MA, Appenzeller C (2010) Risks of model weighting in multimodel climate projections. J Clim 23(15):4175–4191. http://journals.ametsoc.org/doi/abs/10.1175/2010JCLI3594.1
Acknowledgments
We acknowledge the World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP, and we thank the climate modeling groups for producing and making available their model output. For CMIP the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led development of software infrastructure in partnership with the Global Organization for Earth System Science Portals. All CMIP5 data was accessed via the NCI ESGF node. This research was undertaken with the assistance of resources provided at the Australian National University through the National Computational Merit Allocation Scheme supported by the Australian Government.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported in part through the ARC Centre of Excellence in Climate System Science, which is supported by the Australian Commonwealth Government (CE110001028) and by ARC Discovery Grant DP110102618.
Rights and permissions
About this article
Cite this article
Haughton, N., Abramowitz, G., Pitman, A. et al. Weighting climate model ensembles for mean and variance estimates. Clim Dyn 45, 3169–3181 (2015). https://doi.org/10.1007/s00382-015-2531-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-015-2531-3