ALBERT

Description: The second edition of the satellite-derived climate data record CLARA (The CM SAF Cloud, Albedo And Surface Radiation dataset from AVHRR data – second edition denoted as CLARA-A2) is described. The data record covers the 34-year period from 1982 until 2015 and consists of cloud, surface albedo and surface radiation budget products derived from the AVHRR (Advanced Very High Resolution Radiometer) sensor carried by polar-orbiting, operational meteorological satellites. The data record is produced by the EUMETSAT Climate Monitoring Satellite Application Facility (CM SAF) project as part of the operational ground segment. Its upgraded content and methodology improvements since edition 1 are described in detail, as are some major validation results. Some of the main improvements to the data record come from a major effort in cleaning and homogenizing the basic AVHRR level-1 radiance record and a systematic use of CALIPSO-CALIOP cloud information for development and validation purposes. Examples of applications studying decadal changes in Arctic summer surface albedo and cloud conditions are provided.

Description: Cloud top height retrieval from imager instruments is important for nowcasting and for satellite climate data records. A neural network approach for cloud top height retrieval from the imager instrument MODIS (Moderate Resolution Imaging Spectroradiometer) is presented. The neural networks are trained using cloud top layer pressure data from the CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) dataset. Results are compared with two operational reference algorithms for cloud top height: the MODIS Collection 6 Level 2 height product and the cloud top temperature and height algorithm in the 2014 version of the NWC SAF (EUMETSAT (European Organization for the Exploitation of Meteorological Satellites) Satellite Application Facility on Support to Nowcasting and Very Short Range Forecasting) PPS (Polar Platform System). All three techniques are evaluated using both CALIOP and CPR (Cloud Profiling Radar for CloudSat (CLOUD SATellite)) height. Instruments like AVHRR (Advanced Very High Resolution Radiometer) and VIIRS (Visible Infrared Imaging Radiometer Suite) contain fewer channels useful for cloud top height retrievals than MODIS, therefore several different neural networks are investigated to test how infrared channel selection influences retrieval performance. Also a network with only channels available for the AVHRR1 instrument is trained and evaluated. To examine the contribution of different variables, networks with fewer variables are trained. It is shown that variables containing imager information for neighboring pixels are very important. The error distributions of the involved cloud top height algorithms are found to be non-Gaussian. Different descriptive statistic measures are presented and it is exemplified that bias and SD (standard deviation) can be misleading for non-Gaussian distributions. The median and mode are found to better describe the tendency of the error distributions and IQR (interquartile range) and MAE (mean absolute error) are found to give the most useful information of the spread of the errors. For all descriptive statistics presented MAE, IQR, RMSE (root mean square error), SD, mode, median, bias and percentage of absolute errors above 0.25, 0.5, 1 and 2 km the neural network perform better than the reference algorithms both validated with CALIOP and CPR (CloudSat). The neural networks using the brightness temperatures at 11 and 12 µm show at least 32 % (or 623 m) lower MAE compared to the two operational reference algorithms when validating with CALIOP height. Validation with CPR (CloudSat) height gives at least 25 % (or 430 m) reduction of MAE.

Description: The sensitivity in detecting thin clouds of the cloud screening method being used in the CM SAF cloud, albedo and surface radiation data set from AVHRR data (CLARA-A2) cloud climate data record (CDR) has been evaluated using cloud information from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) onboard the CALIPSO satellite. The sensitivity, including its global variation, has been studied based on collocations of Advanced Very High Resolution Radiometer (AVHRR) and CALIOP measurements over a 10-year period (2006–2015). The cloud detection sensitivity has been defined as the minimum cloud optical thickness for which 50 % of clouds could be detected, with the global average sensitivity estimated to be 0.225. After using this value to reduce the CALIOP cloud mask (i.e. clouds with optical thickness below this threshold were interpreted as cloud-free cases), cloudiness results were found to be basically unbiased over most of the globe except over the polar regions where a considerable underestimation of cloudiness could be seen during the polar winter. The overall probability of detecting clouds in the polar winter could be as low as 50 % over the highest and coldest parts of Greenland and Antarctica, showing that a large fraction of optically thick clouds also remains undetected here. The study included an in-depth analysis of the probability of detecting a cloud as a function of the vertically integrated cloud optical thickness as well as of the cloud's geographical position. Best results were achieved over oceanic surfaces at mid- to high latitudes where at least 50 % of all clouds with an optical thickness down to a value of 0.075 were detected. Corresponding cloud detection sensitivities over land surfaces outside of the polar regions were generally larger than 0.2 with maximum values of approximately 0.5 over the Sahara and the Arabian Peninsula. For polar land surfaces the values were close to 1 or higher with maximum values of 4.5 for the parts with the highest altitudes over Greenland and Antarctica. It is suggested to quantify the detection performance of other CDRs in terms of a sensitivity threshold of cloud optical thickness, which can be estimated using active lidar observations. Validation results are proposed to be used in Cloud Feedback Model Intercomparison Project (CFMIP) Observation Simulation Package (COSP) simulators for cloud detection characterization of various cloud CDRs from passive imagery.

Description: A neural-network-based method, quantile regression neural networks (QRNNs), is proposed as a novel approach to estimating the a posteriori distribution of Bayesian remote sensing retrievals. The advantage of QRNNs over conventional neural network retrievals is that they learn to predict not only a single retrieval value but also the associated, case-specific uncertainties. In this study, the retrieval performance of QRNNs is characterized and compared to that of other state-of-the-art retrieval methods. A synthetic retrieval scenario is presented and used as a validation case for the application of QRNNs to Bayesian retrieval problems. The QRNN retrieval performance is evaluated against Markov chain Monte Carlo simulation and another Bayesian method based on Monte Carlo integration over a retrieval database. The scenario is also used to investigate how different hyperparameter configurations and training set sizes affect the retrieval performance. In the second part of the study, QRNNs are applied to the retrieval of cloud top pressure from observations by the Moderate Resolution Imaging Spectroradiometer (MODIS). It is shown that QRNNs are not only capable of achieving similar accuracy to standard neural network retrievals but also provide statistically consistent uncertainty estimates for non-Gaussian retrieval errors. The results presented in this work show that QRNNs are able to combine the flexibility and computational efficiency of the machine learning approach with the theoretically sound handling of uncertainties of the Bayesian framework. Together with this article, a Python implementation of QRNNs is released through a public repository to make the method available to the scientific community.

Description: Cloud retrievals from the Moderate Resolution Imaging Spectroradiometer (MODIS) instruments aboard the satellites Terra and Aqua and the Visible Infrared Imaging Radiometer Suite (VIIRS) instrument aboard the Suomi-NPP satellite are evaluated using a combination of ground-based instruments providing vertical profiles of clouds. The ground-based measurements are obtained from the Atmospheric Radiation Measurement program (ARM) mobile facility, which was deployed in Hyytiälä, Finland, between February and September 2014 for the Biogenic Aerosols – Effects on Clouds and Climate (BAECC) campaign. The satellite cloud parameters cloud top height (CTH) and liquid water path (LWP) are compared with ground-based CTH obtained from a cloud mask created using lidar and radar data and LWP acquired from an a multi-channel microwave radiometer. Clouds from all altitudes in the atmosphere are investigated. The clouds are diagnosed as single or multiple layer using the ground-based cloud mask. For single layer clouds, satellites overestimated CTH by 340 m (16 %) on average. When including multilayer clouds, satellites underestimated CTH by on average 320 m (9 %). MODIS collection 6 overestimated LWP by on average 13 g m−2 (11 %), interestingly, LWP for MODIS collection 5.1 is slightly overestimated by Aqua (4.56 %) but is underestimated by Terra (14.3 %). This underestimation may be attributed to a known issue with a drift in the reflectance bands of the MODIS instrument on Terra. This evaluation indicates that the satellite cloud parameters selected show reasonable agreement with their ground-based counterparts over Finland, with minimal influence from the large solar zenith angle experienced by the satellites in this high latitude location.

Description: The second edition of the satellite-derived climate data record CLARA ("The CM SAF cLoud, Albedo and surface RAdiation dataset from AVHRR data" – second edition denoted CLARA-A2) is described. The data record covers the 34-year period from 1982 until 2015 and consists of cloud, surface albedo and surface radiation budget products derived from the AVHRR (Advanced Very High Resolution Radiometer) sensor carried by polar-orbiting, operational meteorological satellites. The data record is produced by the EUMETSAT Climate Monitoring Satellite Application Facility (CM SAF) project as part of the operational ground segment. Its upgraded content and methodology improvements since edition 1 are described in detail as well as some major validation results. Some of the main improvements of the data record come from a major effort in cleaning and homogenising the basic AVHRR level 1 radiance record and a systematic use of CALIPSO-CALIOP cloud information for development and validation purposes. Examples of applications studying decadal changes in Polar Summer surface albedo and cloud conditions, as well as global cloud redistribution patterns, are provided.

Description: The cloud detection performance of the cloud mask being used in the CM SAF cloud, albedo and surface radiation dataset from AVHRR data (CLARA-A2) cloud climate data record (CDR) has been evaluated in detail using cloud information from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) onboard the CALIPSO satellite. Validation results, including their global distribution, have been calculated from collocations of AVHRR and CALIOP measurements over a ten-year period (2006–2015). The sensitivity of the results to the cloud optical thicknesses of CALIOP-observed clouds were studied leading to the conclusion that the global cloud detection sensitivity (defined as the minimum cloud optical thickness for which 50 % of clouds could be detected) was estimated to 0.225. After applying this optical thickness threshold to the CALIOP cloud mask, results were found to be basically unbiased over most of the globe except over the polar regions where a considerably underestimation of cloudiness could be seen during the polar winter. The probability of detecting clouds in the polar winter could be as low as 50 % over the highest and coldest portions of Greenland and Antarctica, showing that also a large fraction of optically thick clouds remains undetected here. The study included an in-depth analysis of the probability of detecting a cloud as a function of the vertically integrated cloud optical thickness as well as of the cloud’s geographical position. Best results were achieved over oceanic surfaces at mid-to-high latitudes were at least 50 % of all clouds with an optical thickness down to a value of 0.075 were detected. Corresponding cloud detection sensitivities over land surfaces outside of the polar regions were generally larger than 0.2 with maximum values of approximately 0.5 over Sahara and the Arabian Peninsula. For polar land surfaces the values were close to 1 or higher with maximum values of 4.5 over the geographically highest parts of Greenland and Antarctica. The validation method is suggested to be applied also to other satellite-based CDRs and validation results are proposed to be used in Cloud Feedback Model Intercomparison Project (CFMIP) Observation Simulation Package (COSP) simulators for cloud detection characterisation of various cloud CDRs from passive imagery.

Description: Cloud top height retrieval from imager instruments is important for Nowcasting and for satellite climate data records. A neural network approach for cloud top height retrieval from the imager instrument MODIS is presented. The neural networks are trained using cloud top layer pressure data from the CALIOP dataset. Results are compared with two operational reference algorithms for cloud top height: the MODIS Collection 6 level 2 height product and the cloud top temperature and height algorithm (CTTH) in the 2014 version of the NWCSAF Polar Platform System (PPS-v2014). All three techniques are evaluated using both CALIOP and CPR (CloudSat) height. Instruments like AVHRR and VIIRS contain fewer channels useful for cloud top height retrievals than MODIS, therefore several different neural networks are investigated to test how infrared channel selection influences retrieval performance. Also a network with only channels available for the AVHRR1 instrument is trained and evaluated. To examine the contribution of different variables, networks with fewer variables are trained. It is shown that variables containing imager information for neighbouring pixels are very important. Overall results for the neural network height retrievals are very promising. The neural networks using the brightness temperatures at 11 μm and 12 μm show at least 33 % (or 627 m) lower mean absolute error (MAE) compared to the two operational reference algorithms when validating with CALIOP height. Validation with CPR (CloudSat) height gives at least 25 % (or 433 m) reduction of MAE. For the network trained with a channel combination available for AVHRR1, the MAE is at least 542 m better when validated with CALIOP and 414 m when validated with CPR (CloudSat) compared to the two operational reference algorithms. The NWCSAF PPS-2018 release will contain a neural network based cloud height algorithm.

Description: This work is concerned with the retrieval of physical quantities from remote sensing measurements. A neural network based method, Quantile Regression Neural Networks (QRNNs), is proposed as a novel approach to estimate the a posteriori distribution of Bayesian remote sensing retrievals. The advantage of QRNNs over conventional neural network retrievals is that they not only learn to predict a single retrieval value but also the associated, case specific uncertainties. In this study, the retrieval performance of QRNNs is characterized and compared to that of other state-of-the-art retrieval methods. A synthetic retrieval scenario is presented and used as a validation case for the application of QRNNs to Bayesian retrieval problems. The QRNN retrieval performance is evaluated against Markov chain Monte Carlo simulation and another Bayesian method based on Monte Carlo integration over a retrieval database. The scenario is also used to investigate how different hyperparameter configurations and training set sizes affect the retrieval performance. In the second part of the study, QRNNs are applied to the retrieval of cloud top pressure from observations by the moderate resolution imaging spectroradiometer (MODIS). It is shown that QRNNs are not only capable of achieving similar accuracy as standard neural network retrievals, but also provide statistically consistent uncertainty estimates for non-Gaussian retrieval errors. The results presented in this work show that QRNNs are able to combine the flexibility and computational efficiency of the machine learning approach with the theoretically sound handling of uncertainties of the Bayesian framework. Together with this article, a Python implementation of QRNNs is released through a public repository to make the method available to the scientific community.