ALBERT

Description: The climate modelling community has trialled a large number of metrics for evaluating the temporal performance of general circulation models (GCMs), while very little attention has been given to the assessment of their spatial performance, which is equally important. This study evaluated the performance of 36 Coupled Model Intercomparison Project 5 (CMIP5) GCMs in relation to their skills in simulating mean annual, monsoon, winter, pre-monsoon, and post-monsoon precipitation and maximum and minimum temperature over Pakistan using state-of-the-art spatial metrics, SPAtial EFficiency, fractions skill score, Goodman–Kruskal's lambda, Cramer's V, Mapcurves, and Kling–Gupta efficiency, for the period 1961–2005. The multi-model ensemble (MME) precipitation and maximum and minimum temperature data were generated through the intelligent merging of simulated precipitation and maximum and minimum temperature of selected GCMs employing random forest (RF) regression and simple mean (SM) techniques. The results indicated some differences in the ranks of GCMs for different spatial metrics. The overall ranks indicated NorESM1-M, MIROC5, BCC-CSM1-1, and ACCESS1-3 as the best GCMs in simulating the spatial patterns of mean annual, monsoon, winter, pre-monsoon, and post-monsoon precipitation and maximum and minimum temperature over Pakistan. MME precipitation and maximum and minimum temperature generated based on the best-performing GCMs showed more similarities with observed precipitation and maximum and minimum temperature compared to precipitation and maximum and minimum temperature simulated by individual GCMs. The MMEs developed using RF displayed better performance than the MMEs based on SM. Multiple spatial metrics have been used for the first time for selecting GCMs based on their capability to mimic the spatial patterns of annual and seasonal precipitation and maximum and minimum temperature. The approach proposed in the present study can be extended to any number of GCMs and climate variables and applicable to any region for the suitable selection of an ensemble of GCMs to reduce uncertainties in climate projections.

Description: The impacts of climate change on the seasonality of low flows were analysed for 134 sub-catchments covering the River Rhine basin upstream of the Dutch-German border. Three seasonality indices for low flows were estimated, namely the seasonality ratio (SR), weighted mean occurrence day (WMOD) and weighted persistence (WP). These indices are related to the discharge regime, timing and variability in timing of low flow events respectively. The three indices were estimated from: (1) observed low flows; (2) simulated low flows by the semi-distributed HBV model using observed climate as input; (3) simulated low flows using simulated inputs from seven combinations of General Circulation Models (GCMs) and Regional Climate Models (RCMs) for the current climate (1964–2007); (4) simulated low flows using simulated inputs from seven combinations of GCMs and RCMs for the future climate (2063–2098) including three different greenhouse gas emission scenarios. These four cases were compared to assess the effects of the hydrological model, forcing by different climate models and different emission scenarios on the three indices. Significant differences were found between cases 1 and 2. For instance, the HBV model is prone to overestimate SR and to underestimate WP and simulates very late WMODs compared to the estimated WMODs using observed discharges. Comparing the results of cases 2 and 3, the smallest difference was found for the SR index, whereas large differences were found for the WMOD and WP indices for the current climate. Finally, comparing the results of cases 3 and 4, we found that SR decreases substantially by 2063–2098 in all seven sub-basins of the River Rhine. The lower values of SR for the future climate indicate a shift from winter low flows (SR 〉 1) to summer low flows (SR 〈 1) in the two Alpine sub-basins. The WMODs of low flows tend to be earlier than for the current climate in all sub-basins except for the Middle Rhine and Lower Rhine sub-basins. The WP values are slightly larger, showing that the predictability of low flow events increases as the variability in timing decreases for the future climate. From comparison of the error sources evaluated in this study, it is obvious that different RCMs/GCMs have a larger influence on the timing of low flows than different emission scenarios. Finally, this study complements recent analyses of an international project (Rhineblick) by analysing the seasonality aspects of low flows and extends the scope further to understand the effects of hydrological model errors and climate change on three important low flow seasonality properties: regime, timing and persistence.

Description: The impacts of climate change on the seasonality of low flows are analysed for 134 sub-catchments covering the River Rhine basin upstream of the Dutch–German border. Three seasonality indices for low flows are estimated, namely seasonality ratio (SR), weighted mean occurrence day (WMOD) and weighted persistence (WP). These indices are related to the discharge regime, timing and variability in timing of low flow events respectively. The three indices are estimated from: (1) observed low flows; (2) simulated low flows by the semi distributed HBV model using observed climate; (3) simulated low flows using simulated inputs from seven climate scenarios for the current climate (1964–2007); (4) simulated low flows using simulated inputs from seven climate scenarios for the future climate (2063–2098) including different emission scenarios. These four cases are compared to assess the effects of the hydrological model, forcing by different climate models and different emission scenarios on the three indices. The seven climate scenarios are based on different combinations of four General Circulation Models (GCMs), four Regional Climate Models (RCMs) and three greenhouse gas emission scenarios. Significant differences are found between cases 1 and 2. For instance, the HBV model is prone to overestimate SR and to underestimate WP and simulates very late WMODs compared to the estimated WMODs using observed discharges. Comparing the results of cases 2 and 3, the smallest difference is found in the SR index, whereas large differences are found in the WMOD and WP indices for the current climate. Finally, comparing the results of cases 3 and 4, we found that SR has decreased substantially by 2063–2098 in all seven subbasins of the River Rhine. The lower values of SR for the future climate indicate a shift from winter low flows (SR 〉 1) to summer low flows (SR 〈 1) in the two Alpine subbasins. The WMODs of low flows tend to be earlier than for the current climate in all subbasins except for the Middle Rhine and Lower Rhine subbasins. The WP values are slightly larger, showing that the predictability of low flow events increases as the variability in timing decreases for the future climate. From comparison of the uncertainty sources evaluated in this study, it is obvious that the RCM/GCM uncertainty has the largest influence on the variability in timing of low flows for future climate.

Description: This paper investigates the skill of 90-day low-flow forecasts using two conceptual hydrological models and one data-driven model based on Artificial Neural Networks (ANNs) for the Moselle River. The three models, i.e. HBV, GR4J and ANN-Ensemble (ANN-E), all use forecasted meteorological inputs (precipitation P and potential evapotranspiration PET), whereby we employ ensemble seasonal meteorological forecasts. We compared low-flow forecasts for five different cases of seasonal meteorological forcing: (1) ensemble P and PET forecasts; (2) ensemble P forecasts and observed climate mean PET; (3) observed climate mean P and ensemble PET forecasts; (4) observed climate mean P and PET and (5) zero P and ensemble PET forecasts as input for the models. The ensemble P and PET forecasts, each consisting of 40 members, reveal the forecast ranges due to the model inputs. The five cases are compared for a lead time of 90 days based on model output ranges, whereas the models are compared based on their skill of low-flow forecasts for varying lead times up to 90 days. Before forecasting, the hydrological models are calibrated and validated for a period of 30 and 20 years respectively. The smallest difference between calibration and validation performance is found for HBV, whereas the largest difference is found for ANN-E. From the results, it appears that all models are prone to over-predict runoff during low-flow periods using ensemble seasonal meteorological forcing. The largest range for 90-day low-flow forecasts is found for the GR4J model when using ensemble seasonal meteorological forecasts as input. GR4J, HBV and ANN-E under-predicted 90-day-ahead low flows in the very dry year 2003 without precipitation data. The results of the comparison of forecast skills with varying lead times show that GR4J is less skilful than ANN-E and HBV. Overall, the uncertainty from ensemble P forecasts has a larger effect on seasonal low-flow forecasts than the uncertainty from ensemble PET forecasts and initial model conditions.

Description: This paper investigates the skill of 90 day low flow forecasts using two conceptual hydrological models and two data-driven models based on Artificial Neural Networks (ANNs) for the Moselle River. One data-driven model, ANN-Indicator (ANN-I), requires historical inputs on precipitation (P), potential evapotranspiration (PET), groundwater (G) and observed discharge (Q), whereas the other data-driven model, ANN-Ensemble (ANN-E), and the two conceptual models, HBV and GR4J, use forecasted meteorological inputs (P and PET), whereby we employ ensemble seasonal meteorological forecasts. We compared low flow forecasts without any meteorological forecasts as input (ANN-I) and five different cases of seasonal meteorological forcing: (1) ensemble P and PET forecasts; (2) ensemble P forecasts and observed climate mean PET; (3) observed climate mean P and ensemble PET forecasts; (4) observed climate mean P and PET and (5) zero P and ensemble PET forecasts as input for the other three models (GR4J, HBV and ANN-E). The ensemble P and PET forecasts, each consisting of 40 members, reveal the forecast ranges due to the model inputs. The five cases are compared for a lead time of 90 days based on model output ranges, whereas the four models are compared based on their skill of low flow forecasts for varying lead times up to 90 days. Before forecasting, the hydrological models are calibrated and validated for a period of 30 and 20 years respectively. The smallest difference between calibration and validation performance is found for HBV, whereas the largest difference is found for ANN-E. From the results, it appears that all models are prone to over-predict low flows using ensemble seasonal meteorological forcing. The largest range for 90 day low flow forecasts is found for the GR4J model when using ensemble seasonal meteorological forecasts as input. GR4J, HBV and ANN-E under-predicted 90 day ahead low flows in the very dry year 2003 without precipitation data, whereas ANN-I predicted the magnitude of the low flows better than the other three models. The results of the comparison of forecast skills with varying lead times show that GR4J is less skilful than ANN-E and HBV. Furthermore, the hit rate of ANN-E is higher than the two conceptual models for most lead times. However, ANN-I is not successful in distinguishing between low flow events and non-low flow events. Overall, the uncertainty from ensemble P forecasts has a larger effect on seasonal low flow forecasts than the uncertainty from ensemble PET forecasts and initial model conditions.

Description: The process of model evaluation is not only an integral part of model development and calibration but also of paramount importance when communicating modelling results to the scientific community and stakeholders. The modelling community has a large and well tested toolbox of metrics to evaluate temporal model performance. On the contrary, spatial performance evaluation is not corresponding to the grand availability of spatial observations readily available and to the sophisticate model codes simulating the spatial variability of complex earth system processes. This study makes a contribution towards advancing spatial pattern oriented model calibration by rigorously testing a multiple-component performance metric. The promoted SPAtial EFficiency (SPAEF) metric reflects three equally weighted components: correlation, coefficient of variation and histogram overlap. This multiple-component approach is found to be advantageous in order to achieve the complex task of comparing spatial patterns. SPAEF, its three components individually and two alternative spatial performance metrics, i.e. connectivity analysis and fractions skill score, are applied in a spatial pattern oriented model calibration of a catchment model in Denmark. Results suggest the importance of multiple-component metrics, because stand-alone metrics tend to fail to provide holistic pattern information to the optimizer. The three SPAEF components are found to be independent which allows them to complement each other in a meaningful way. In order to optimally exploit spatial observations made available by remote sensing platforms this study suggests applying bias insensitive metrics which further allow comparing variables which are related but may differ in unit. This study applies SPAEF in the hydrological context using the mesoscale Hydrologic Model (mHM; version 5.6), but we see great potential across disciplines related to spatial distributed earth system modelling.

Description: Satellite-based earth observations offer great opportunities to improve spatial model predictions by means of spatial-pattern-oriented model evaluations. In this study, observed spatial patterns of actual evapotranspiration (AET) are utilised for spatial model calibration tailored to target the pattern performance of the model. The proposed calibration framework combines temporally aggregated observed spatial patterns with a new spatial performance metric and a flexible spatial parameterisation scheme. The mesoscale hydrologic model (mHM) is used to simulate streamflow and AET and has been selected due to its soil parameter distribution approach based on pedo-transfer functions and the build in multi-scale parameter regionalisation. In addition two new spatial parameter distribution options have been incorporated in the model in order to increase the flexibility of root fraction coefficient and potential evapotranspiration correction parameterisations, based on soil type and vegetation density. These parameterisations are utilised as they are most relevant for simulated AET patterns from the hydrologic model. Due to the fundamental challenges encountered when evaluating spatial pattern performance using standard metrics, we developed a simple but highly discriminative spatial metric, i.e. one comprised of three easily interpretable components measuring co-location, variation and distribution of the spatial data. The study shows that with flexible spatial model parameterisation used in combination with the appropriate objective functions, the simulated spatial patterns of actual evapotranspiration become substantially more similar to the satellite-based estimates. Overall 26 parameters are identified for calibration through a sequential screening approach based on a combination of streamflow and spatial pattern metrics. The robustness of the calibrations is tested using an ensemble of nine calibrations based on different seed numbers using the shuffled complex evolution optimiser. The calibration results reveal a limited trade-off between streamflow dynamics and spatial patterns illustrating the benefit of combining separate observation types and objective functions. At the same time, the simulated spatial patterns of AET significantly improved when an objective function based on observed AET patterns and a novel spatial performance metric compared to traditional streamflow-only calibration were included. Since the overall water balance is usually a crucial goal in hydrologic modelling, spatial-pattern-oriented optimisation should always be accompanied by traditional discharge measurements. In such a multi-objective framework, the current study promotes the use of a novel bias-insensitive spatial pattern metric, which exploits the key information contained in the observed patterns while allowing the water balance to be informed by discharge observations.

Description: The climate modelling community has trialled a large number metrics to evaluate the temporal performance of the Global Circulation Models (GCMs) for the selection of GCMs, while very little attention has been given to spatial performance of GCMs which is equally important. This study evaluated the performance of 20 Coupled Model Intercomparison Project 5 (CMIP5) GCMs pertaining to their skills in simulating mean annual, monsoon and winter precipitation over Pakistan using state-of-the-art spatial metrics; SPAtial EFficiency, Goodman–Kruskal's lambda, Fractions Skill Score, Cramer’s V, Mapcurves, and Kling-Gupta efficiency for the period 1961–2005. The multi-model ensemble (MME) precipitation was generated through intelligent merging of simulated precipitation of selected GCMs employing Random Forest (RF) regression and Simple Mean (SM). The results indicated some differences in the ranks of GCMs for different metrics. The overall ranks indicated NorESM1-M, CESM1-CAM5, GFDL-CM3 and GFDL-ESM2G as the best GCMs in simulating the spatial patterns of mean annual, monsoon and winter precipitation over Pakistan. MME precipitation generated based on the best performing GCMs showed more similarities with observed precipitation compared to precipitation simulated by individual GCMs. The MME developed using RF displayed better performance than the MME-based on SM. Multiple spatial metrics have been used for the first time for selecting GCMs based on their capability to mimic the spatial patterns of annual and seasonal precipitation. The approach suggested in the present study can be extended to any number of GCMs and climate variables and applicable to any region for the suitable selection of an ensemble of GCMs to reduce uncertainties in climate projections.

Description: Satellite based earth observations offer great opportunities to improve spatial model predictions by means of spatial pattern oriented model evaluations. In this study, observed spatial patterns of actual evapotranspiration (AET) are utilized for spatial model calibration tailored to target the pattern performance of the model. The proposed calibration framework combines temporally aggregated observed spatial patterns with a new spatial performance metric and a flexible spatial parameterisation scheme. The mesoscale Hydrologic Model (mHM) is used to simulate streamflow and AET and has been selected due to its soil parameter distribution approach based on pedotransfer functions and the build in multiscale parameter regionalization. In addition two new domain specific spatial parameter distribution options have been incorporated in the model in order to increase the flexibility of root fraction coefficient and potential evapotranspiration correction parameterisations, based on soil type and vegetation density. These parametrisations are utilized as they are most relevant for simulated AET patterns from the hydrologic model. Due to the fundamental challenges encountered when evaluating spatial pattern performance using standard metrics, we developed a simple but highly discriminative spatial metric i.e. comprised of three easily interpretable components measuring co-location, variation and distribution of the spatial data. The study shows that with flexible spatial model parameterisation used in combination with the appropriate objective functions, the simulated spatial patterns of actual evapotranspiration become substantially more similar to the satellite based estimates. Overall 26 parameters are identified for calibration through a sequential screening approach based on a combination of streamflow and spatial pattern metrics. The robustness of the calibrations is tested using an ensemble of nine calibrations based on different seed numbers using the shuffled complex evolution optimizer. The calibration results reveal a limited trade-offs between streamflow dynamics and spatial patterns illustrating the benefit of combining separate observation types and objective functions. At the same time, the simulated spatial patterns of AET significantly improved when including an objective function based on observed AET patterns and a novel spatial performance metric compared to traditional streamflow only calibration. Since the overall water balance is usually a crucial goal in the hydrologic modelling, spatial pattern oriented optimization should always be accompanied by traditional discharge measurements. In such a multi-objective framework, the current study promotes the use of a novel bias-insensitive spatial pattern metric, which exploits the key information contained in the observed patterns while allowing the water balance to be informed by discharge observations.

Description: The process of model evaluation is not only an integral part of model development and calibration but also of paramount importance when communicating modelling results to the scientific community and stakeholders. The modelling community has a large and well-tested toolbox of metrics to evaluate temporal model performance. In contrast, spatial performance evaluation does not correspond to the grand availability of spatial observations readily available and to the sophisticate model codes simulating the spatial variability of complex hydrological processes. This study makes a contribution towards advancing spatial-pattern-oriented model calibration by rigorously testing a multiple-component performance metric. The promoted SPAtial EFficiency (SPAEF) metric reflects three equally weighted components: correlation, coefficient of variation and histogram overlap. This multiple-component approach is found to be advantageous in order to achieve the complex task of comparing spatial patterns. SPAEF, its three components individually and two alternative spatial performance metrics, i.e. connectivity analysis and fractions skill score, are applied in a spatial-pattern-oriented model calibration of a catchment model in Denmark. Results suggest the importance of multiple-component metrics because stand-alone metrics tend to fail to provide holistic pattern information. The three SPAEF components are found to be independent, which allows them to complement each other in a meaningful way. In order to optimally exploit spatial observations made available by remote sensing platforms, this study suggests applying bias insensitive metrics which further allow for a comparison of variables which are related but may differ in unit. This study applies SPAEF in the hydrological context using the mesoscale Hydrologic Model (mHM; version 5.8), but we see great potential across disciplines related to spatially distributed earth system modelling.