Evaluation of SMOS, SMAP, ASCAT and Sentinel-1 Soil Moisture Products at Sites in Southwestern France

Abstract

This study evaluates the accuracy of several recent remote sensing Surface Soil Moisture (SSM) products at sites in southwestern France. The products used are Soil Moisture Active Passive “SMAP” (level 3: 36 km × 36 km, level 3 enhanced: 9 km × 9 km, and Level 2 SMAP/Sentinel-1: 1 km × 1km), Advanced Scatterometer “ASCAT” (level 2 with three spatial resolution 25 km × 25 km, 12.5 km × 12.5 km, and 1 km × 1 km), Soil Moisture and Ocean Salinity “SMOS” (SMOS INRA-CESBIO “SMOS-IC”, SMOS Near-Real-Time “SMOS-NRT”, SMOS Centre Aval de Traitement des Données SMOS level 3 “SMOS-CATDS”, 25 km × 25 km) and Sentinel-1(S1) (25 km × 25 km, 9 km × 9 km, and 1 km × 1 km). The accuracy of SSM products was computed using in situ measurements of SSM observed at a depth of 5 cm. In situ measurements were obtained from the SMOSMANIA ThetaProbe (Time Domaine reflectometry) network (7 stations between 1 January 2016 and 30 June 2017) and additional field campaigns (near Montpellier city in France, between 1 January 2017 and 31 May 2017) in southwestern France. For our study sites, results showed that (i) the accuracy of the Level 2 SMAP/Sentinel-1 was lower than that of SMAP-36 km and SMAP-9 km; (ii) the SMAP-36 km and SMAP-9 km products provide more precise SSM estimates than SMOS products (SMOS-IC, SMOS-NRT, and SMOS-CATDS), mainly due to higher sensitivity of SMOS to RFI (Radio Frequency Interference) noise; and (iii) the accuracy of SMAP-36 km and SMAP-9 km products was similar to that of ASCAT (ASCAT-25 km, ASCAT-12.5 km and ASCAT-1 km) and S1 (S1-25 km, S1-9 km, and S1-1 km) products. The accuracy of SMAP, Sentinel-1 and ASCAT SSM products calculated using the average of statistics obtained on each site is defined by a bias of about −3.2 vol. %, RMSD (Root Mean Square Difference) about 7.6 vol. %, ubRMSD (unbiased Root Mean Square Difference)about 5.6 vol. %, and R coefficient about 0.57. For SMOS products, the station average bias, RMSD, ubRMSD, and R coefficient were about −10.6 vol. %, 12.7 vol. %, 5.9 vol. %, and 0.49, respectively.

1. Introduction

Understanding and simulating water cycle behavior allows forecasting of important natural phenomena such as drought, flood, climate change, and landslides [1]. To better understand the workings of the terrestrial water cycle, monitoring the spatial and temporal variations of the Surface Soil Moisture (SSM) is crucial because the SSM plays a principal role in the partitioning of rainfall into runoff and infiltration [2,3,4,5,6]. Several satellite missions and instruments including SMOS (Soil Moisture and Ocean Salinity) [7,8], SMAP (Soil moisture Active and Passive) [9], and ASCAT (Advanced Scatterometer) [10] provide SSM estimates at coarse spatial resolution and very high revisit time (up to 1 day). Recently, SSM maps at very high spatial resolution (up to plot scale) and high revisit time (6 days) derived from Sentinel-1 (S1) satellite were provided for the Occitanie region in the southern part of France [11].

SMOS mission [7,8] was launched in 2009 as the second Earth Explorer Opportunity Mission initiative from European Space Agency (ESA) with contributions of the Centre National d’Etudes Spatiales (CNES), and the Centro para el Desarrollo Teccnologico Industrial (CDTI). It is the first satellite dedicated to SSM retrievals over continental surfaces and Seas Surface Salinity (SSS) over the oceans using an L-band interferometric radiometer at 1.4 GHz. SMOS has a sun-synchronous orbit at 757 km altitude with a 06:00 LST (Local Solar Time) ascending equator crossing time and an 18:00 LST descending equator crossing time. SMOS provides a global coverage twice every three days with multi-incidence-angle observations at full polarization across a 900 km swath. Radio Frequency Interference (RFI) pollutes part of the SMOS data several regions of the globe are impacted [12].

SMAP [9] satellite was developed by NASA (National Aeronautics and Space Administration) and launched on 31 January 2015 (spatial coverage: 60°N−60°S, 180°E−180°W). SMAP was placed into a polar Sun-synchronous ascending (overpass at ~6:00 PM LST) and descending (overpass at ~6:00 AM LST) orbit. SMAP carries L-band radar (1.26 GHz) and radiometer (1.41 GHz) instruments. The radar instrument is ceased on 7 July 2015 while the L-band radiometer continues functioning as designed. These products are made available publicly through two NASA-designated data centers, Alaska Satellite Facility (ASF) and National Snow and Ice Data Center (NSIDC).

ASCAT sensor is a real aperture radar providing data with a spatial resolution of 25 km × 25 km and revisit time of 1–3 day [10,13,14]. The prime objective of ASCAT is to measure wind speed and direction over the oceans. Two identical ASCAT are operational on board of Metop-A (launched on 19 October 2006) and Metop-B (launched on 17 September 2012) satellites. Metop is a series of three sun-synchronous polar orbiting meteorological satellites developed by the European Space Agency (ESA) and operated by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT). Metop flies at an altitude of about 800 km above the earth’s surface and needs about 101 min to complete one orbit. The ASCAT local equator crossing time is about 09:30 in descending orbit and 21:30 in ascending orbit. ASCAT SSM products have been developed by the Department of Geodesy and Geoinformation of the Vienna University of Technology and the data service is provided in partnership with the EUMETSAT’s Satellite Application Facility (SAF) on support to Operational Hydrology and Water Management (H-SAF).

The S1 mission from ESA in the framework of the Copernicus program is a constellation of polar-orbiting SAR (Synthetic Aperture Radar) satellites operating at C-band (~5.4 GHz). The Sentinel-1A (S1A) and Sentinel-1B (S1B) satellites were launched on 03 April 2014 and 22 April 2016, respectively. The S1A and S1B SAR sensors operate in four acquisition modes: Stripmap (SM), Interferometric Wide swath (IW), Extra-Wide swath (EW), Wave (WV). SM, IW, and EW are available in single (HH or VV) or dual polarization (HH + HV or VV + VH). WV operates only in single polarization (HH or VV). The primary conflict-free mode is IW (VV + VH) over land. This IW mode provides images with a spatial resolution of ~5 m × 20 m (pixel spacing of 10 m × 10 m) and revisit time of 6 days over the equator with both satellites. Recently, El Hajj et al. [11] developed an operational method based on the synergic use of S1 and S2 data for soil moisture mapping on agricultural plots at very high spatial resolution (up to plot scale).

The objective of this study is to evaluate the accuracy of SMAP, SMOS, ASCAT and S1 SSM products over the SMOSMANIA sites in southwestern France. Numerous studies have evaluated the robustness of SMAP, SMOS, and ASCAT SSM products at regional and global scale [15,16,17,18,19,20,21,22,23,24,25,26,27]. This paper completes these studies by evaluating in addition the two recent products; the Level 2 SMAP/S1 product with spatial resolution of 1 km × 1 km and the high resolution SSM maps derived from S1 satellites. The evaluation of products was realized through in situ SSM measurements observed at a depth of 5 cm. Section 2 describes the used SSM products and in situ SSM measurements. The results and discussions are provided in Section 3 and Section 4. Finally, Section 5 presents the main conclusions.

2. Dataset Description

In this study, SSM products of SMAP, ASCAT, SMOS, and S1 as well as in situ SSM measurements were used. For all SSM products, pixels which partially cover the sea were eliminated.

2.1. In Situ Soil Moisture Measurements

SMOSMANIA [28,29] is a long-term effort to acquire profiles of soil moisture from 21 automated weather stations in southwestern and southeastern France. Based on the existing automatic weather station network of Météo-France, SMOSMANIA was developed to validate remote sensing and model soil moisture estimates. The stations were chosen to form a Mediterranean–Atlantic transect following the marked climatic gradient between the two coastlines. The three most eastward stations are representative of a Mediterranean climate. Four soil moisture probes were horizontally installed per station at four depths: 5, 10, 20, and 30 cm. The ThetaProbe ML2X of Delta-T Device was chosen because it has been used successfully during previous long-term campaigns of Meteo-France and because it can easily be interfaced with the automatic stations. SMOSMANIA provides measurements in m³m⁻³ with a 12-min time step. Twelve stations (chosen to form a Mediterranean-Atlantic transect following the marked climatic gradient between the two coastlines) provide measurements from 2006 in southwestern France. The network has been extended in 2009 with nine new stations located in southeastern France [24]. During the installation of soil moisture probes, soil samples were collected at the four depths of the soil moisture profile to calibrate the probe. The soil characteristics of the stations can be found in Albergel et al. [29] and Parrens et al. [24]. The observations from this well-monitored network have been extensively used for the validation of modeled and satellite-derived soil moisture, including ASCAT and SMOS (e.g., [16,18,24,30]). In this study, the SMOSMANIA twelve stations located in southwestern of France were considered. Among these twelve stations only 7 stations (NBN, CRD, MTM, PRG, SFL, SVN, and URG) were used (Figure 1). Four stations (LZC, MNT, LHS, and CDM) were eliminated because a comparison of their recorded SSM and rainfall events show illogical behavior. For instance, the MNT station records between 25 November 2016 and 21 December 2016 were stable and very high (about 32 vol. %) while non-important precipitation events were recorded (cumulative rainfall of 5.8 mm between 25 November 2016 and 21 December 2016). It should be noted that one station (SBR) was eliminated because the S1 SSM product is not available for this station located far from the Occitanie region in France. Comprehensive descriptions of the SMOSMANIA sites can be found in [24,29].

In addition to SMOSMANIA network measurements, the SSM was measured between 1 January 2017 and 31 May 2017 in 23 reference plots (10 grasslands and 13 wheat) distributed in area of 12 km × 12 km near Montpellier, France [11]. For these plots, the climate is Mediterranean with a rainy season between mid-October and March, the average cumulative rainfall is approximately 750 mm, the average air temperature varies between 2.9 °C and 29.3 °C, and the top soil texture of the agricultural fields is a loam. For each reference plot, twenty-five to thirty measurements of volumetric soil moisture were conducted in the top 5 cm of soil by means of a calibrated TDR (Time Domain Reflectometry) probe.

2.2. SMOS-Centre Aval de Traitement des Données SMOS Level 3 product

The SMOS Level 3 SSM (RE04v300) products [31] was produced by the Centre Aval de Traitement des Données SMOS (CATDS). The data are presented over the Equal-Area Scalable Earth (EASE grid 2) [32] with a sampling of 25 km × 25 km. The L3 CATDS are based on a multi-orbit retrieval of soil moisture considering a correlated vegetation optical depth over a period of 7 days (maximum 3 revisits). As the Level 2 [25] algorithm, the Level 3 one uses the L-band Microwave Emission of the Biosphere (L-MEB) [33] Radiative Transfer (RT) model. The contributions of 4 km unitary units are convoluted with the antenna pattern function. The European Center for Metrological Weather Forecast (ECMWF) forecasted surface temperature is used in the forward RT model. The retrieved SSM data are filtered using the Data Quality Index (DQX) provided in the product (DQX < 0.06 m³/m³) based on previous studies [15,23]. SMOS-CATDS were downloaded through the CATDS website (https://www.catds.fr/). In this study, the used SMOS-CATDS soil moisture products are acquired between 1 January 2016 and 30 Jun 2017.

2.3. SMOS Near-Real-Time Product

The SMOS Near-Real-Time (SMOS-NRT) SSM dataset is an ESA product designed to provide SSM in less than 3.5 h after sensing. The SMOS NRT SSM algorithm [34] uses a neural network (NN) trained using SMOS Level 2 SSM [35]. The input data for the NN are SMOS brightness temperatures (TB) with incidence angles from 30° to 45° for horizontal and vertical polarizations and soil temperature in the 0–7 cm layer from the European Centre for Medium-Range Weather Forecasts (ECMWF) models. The swath-width of the NRT SSM dataset is slightly lower than that of the original Level 2 product (~915 km instead of ~1150 km) due to the incidence angle range requirement. The SMOS-NRT SSM dataset shows similar but somewhat better performances than the original Level 2 SSM dataset when compared to in situ measurements from two national networks spanning all over the USA [34]; the SCAN (Soil Climate Analysis Network) [36] and USCRN (US Climate Reference Network) [37] ThetaProbe networks. SMOS NRT SSM data are available from the ESA SMOS web portal and via the broadcast service of the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) (https://www.eumetsat.int/website/home/index.html). The data version used in this study is version 100. SMOS-NRT data were masked out to remove grid cells with relative SSM error above 30%. In this study, the used SMOS-NRT soil moisture products are between 1 January 2016 and 30 Jun 2017.

2.4. SMOS INRA-CESBIO Product

The SMOS INRA-CESBIO (SMOS-IC) algorithm corresponds to the original SMOS retrieval algorithm: namely the two-parameter inversion of the L-MEB model (L-band Microwave Emission of the Biosphere) as defined in [33]. SMOS-IC is different from the operational SMOS Level 2 and Level 3 algorithms in several ways: (i) it is as much as possible independent of auxiliary data which contain errors that may propagate into noise and biases in the retrievals. For instance, contrary to the SMOS L2 and L3 algorithms, SMOS-IC does not require normalized SSM data from ECMWF and Leaf Area Index (LAI) as inputs (ii) it is based on a top-down approach considering the pixel as homogeneous, contrary to L2 and L3 which are based on a complex bottom-up approach, in which the TB is simulated using a detailed description of the SMOS footprint at a resolution of 4 km × 4 km. Recent inter-comparisons in terms of both SSM and VOD (Vegetation Optical Depth) showed SMOS-IC retrievals performed very well (see Wigneron et al. [38] for a review). SMOS-IC provides global gridded (EASE grid 2) daily SSM (m³/m³) and Vegetation Optical depth (VOD) in the Network Common Data Form (NetCDF) format with a ~25 km cylindrical projection (ascending and descending overpasses at 06:00 a.m. and 06:00 p.m. Local Solar Time, respectively). The SMOS-IC products will be publicly available at CATDS (https://www.catds.fr/). For more details on SMOS-IC products and its algorithm, the reader is referred to [38,39,40]. In this study, the used SMOS-IC soil moisture products are acquired between 1 January 2016 and 30 June 2017. SMOS-IC data were masked out to remove grid cells with brightness temperature error higher than 8 K.

2.5. Soil Moisture Active Passive Products

The daily global SMAP SSM products were generated by the SMAP Science Data Processing System (SDS) at JPL (Jet Propulsion Laboratory). The SDS provides the Level-3 SMAP SSM (L3_SSM) with a spatial resolution of 36 km × 36 km, the Level-3 SMAP Enhanced SSM products (L3_SSM_E) with a spatial resolution of 9 km × 9 km, and the Level-2 SMAP/S1 SSM products (L2_SSM) with a spatial resolution of 1 km × 1 km.

For the 36 km L3_SSM, the soil moisture is derived mainly from the SMAP Level-1C TB (L1C_TB) product that contains the time-ordered, geolocated, calibrated L1B_TB brightness temperatures which have been resampled to the fixed 36-km EASE2 grid. The retrieval of the soil moisture is based on the tau-omega model which is an approximation to the radiative transfer equation [41]. In the tau-omega model, the attenuation of the soil TB emission by the vegetation layer is considered according to method developed in [42]. To obtain accurate soil moisture estimation, several ancillary datasets were used. These ancillary data sets include surface temperature, vegetation opacity, vegetation single scattering albedo, surface roughness information, land cover type classification, soil texture, and data flags for identification of land, water, precipitation, RFI, urban areas, mountainous terrain, permanent ice/snow, and dense vegetation [41].

The 9 km L3_SSM_E is derived mainly from the Level 1C TB Enhanced (L1C_TB_E) using the same algorithm as the 36 km L3_SSM. After SMAP radar instrument stopped functioning in July 2015, the SMAP project take advantage of the SMAP radiometer oversampling on orbit to generates the Level-1B Enhanced TB dataset (L1B_TB_E) by using a Backus-Gilbert interpolation approach. The L1B_TB_E are provided on a 9 km EASE2 grid and used to produce the L1C_TB_E.

The 1 km SMAP/S1 (L2_SSM) is derived using the S1A and S1B radar backscatter and the 9 km enhanced radiometer TB available in the SMAP L3_SSM_E products. The L2_SSM product uses the S1A and S1B SAR data to disaggregate SMAP L-band radiometer measurements from the ~40 km radiometer measurement to a 3 and 9 km gridded product. Soil moisture retrievals are performed on both resolution of 3 km × 3 km and 1 km × 1 km.

In this study, the three SMAP SSM products (L3_SSM, L3_SSM_E, and L2_SSM) were acquired between 1 January 2016 and 30 June 2017. SMAP SSM data were eliminated when the soil temperature was below 273.15 K, the SSM was lower than 2 vol. % and higher than 50 vol. %, and the flag for the freeze/thaw fraction indicated an unfrozen soil [43]. The SMAP products (L3_SSM, L3_SSM_E, and L2_SSM) were downloaded from the “EARTHDATA” website (https://search.earthdata.nasa.gov/search).

2.6. Advanced Scatterometer Products

The Advanced Scatterometer (ASCAT) radar is one of the instruments carried by ESA’s Metop satellites. It operates in the C-band (5.3 GHz) and in the vertical polarization (VV). Over land, the measured radar backscattering coefficient depends on the soil moisture, surface roughness, vegetation characteristics and the incidence angle of the transmitted radar beam. The surface soil moisture data is retrieved from the backscattering coefficient, using a change detection method developed at the Institute of Photogrammetry and Remote Sensing (IPF), Vienna University of Technology (TU Wien), and described by [44,45,46]. The relative soil moisture data, ranging between 0% and 100%, are derived by scaling the normalized backscattering coefficients at 40° degree incidence angle between the lowest/highest values corresponding to the driest/wettest soil conditions [10,47] Equation (1). The derived soil moisture product, expressed in relative units and referred to as “Surface Soil Moisture” (SSM), represents the water content in the first 5 cm of the soil and ranges between the extremes corresponding to totally dry conditions, and a totally saturated water capacity. In order to compare SSM with ground measurements, SSM products were converted to physical units of m³m⁻³ by using the 90% confidence interval of a Gaussian distribution [47] equal to μ ± 1.65 σ, where μ and σ are respectively the mean and the standard deviation of the theta probe ground data:

$$\mathrm{SSM}\left(t\right)=m_s\left(t\right)*\left({\mathrm{SSM}}_{\max }-{\mathrm{SSM}}_{\min }\right)+{\mathrm{SSM}}_{\min }$$

(1)

where SSM(t) is the surface soil moisture content at a time t [m³m⁻³], m_s(t) is the ASCAT scatterometer surface soil moisture at a time t, SSM_max is the maximum wetness value [m³m⁻³] equal to μ + 1.65 σ and SSM_min is the minimum wetness value [m³m⁻³] equal to μ − 1.65 σ. From our in situ SSM dataset, SSM_max = 39.7 vol. % and SSM_min = 7.5 vol. %.

In this study, three daily ASCAT products with spatial resolution of 25 km × 25 km (H102 and H103), 12.5 km × 12.5 km (H101 and H16), and 1 km × 1 km (H08) were used. ASCAT-25 km and ASCAT-12.5 km were generated with the WARP-NRT software provided by TU-Wien [10]. The ASCAT-1 km product is a result of combining ASCAT-25 km (H102 and H103) data with 1 km backscatter information derived from ENVISAT ASAR (years 2004–2012) and S1 to better fit hydrological requirements. All ASCAT products are between 1 January 2016 and 30 June 2017. ASCAT-25 km and ASCAT-12.5 km were downloaded from the EUMESTAT website (https://www.eumetsat.int/website/home/index.html) and H08 product was downloaded from the H-SAF website (http://hsaf.meteoam.it/soil-moisture.php?tab=1). ASCAT data were screened to remove grid cells with SSM error higher than 10 vol. %, topographic complexity higher than 20%, probable fraction of frozen soil higher than 5%, and probable inundation or wetland fraction higher than 5% [43,48].

2.7. Sentinel-1 Products

The high resolution (up to plot scale) SSM maps for agricultural areas were derived from S1 SAR images [11]. To map the soil moisture in agriculture areas, El Hajj et al. [11] used a land cover map [49]. The soil moisture maps were generated by coupling S1 and S2 data. The retrieval algorithm uses the NN technique to invert the radar signal and estimate the soil moisture. The retrieval algorithm uses the WCM (Water Cloud Model) [50] combined with the IEM (Integral Equation Model) [51] to account for vegetation contribution (direct scattering and attenuation) on the total backscatter radar signal. First, a parametrized WCM [52] combined with the modified IEM [53,54,55] were used to generate a synthetic database of radar backscattering coefficient in the VV polarization (incidence angle between 20° and 45°) for a wide range of soil moisture (2 < SSM (vol. %) < 40), soil roughness (0.5 < Hrms (cm) < 3.8), and vegetation conditions (0 < NDVI < 0.75). Second, the simulated backscattering coefficients and the NDVI values were noisy to obtain a more realistic synthetic database. Third, the NN was trained using half of the noisy synthetic database and validated using the other half. Finally, the trained neural network was applied to S1 and S2 data to derive the soil moisture. To improve the soil moisture estimates, a priori knowledge about the soil moisture state is introduced. This a priori knowledge was defined based on a precipitation record [56]. The integration of a priori information constrains the range of possible soil moisture parameter values estimated by the NNs and thus leads to a better estimation of the soil moisture.

In this study, the high resolution SSM maps were derived from S1 images acquired at ~06:00 p.m. (UTC time) [11]. These maps cover the all Occitanie region of France (Figure 1) with high revisit time (6 days). To compare with SMAP ASCAT and SMOS, coarse resolution SSM maps with a spatial resolution of 25 km × 25 km, 9 km × 9 km, and 1 km × 1 km were computed from the very high spatial resolution soil moisture maps by averaging pixels of high resolution soil moisture maps values within a grid cell. The S1 derived SSM maps are available on the Theia/Geosud website (http://ids.equipex-geosud.fr/web/guest/humidite-de-sol-radar-s1-a/b).

3. Results

To evaluate SMAP, ASCAT, SMOS, and S1 SSM products, the SSM from these products were compared to in situ SSM measurements. The accuracy of SSM products was determined by means of the bias (SSM product – in situ SSM), the Root Mean Square Difference (RMSD), the unbiased RMSD (ubRMSD = $\sqrt{RMS{D}^2-bia{s}^2}$), and the Pearson correlation coefficient (R). For each product, these statistic variables were computed using: (1) together all the data from all the stations, (2) the data of each station, and (3) the average of statistics obtained in each station alone (station average bias, RMSD, ubRMSD, and R).

Soil moisture time series usually show a strong seasonal pattern possibly artificially increasing the perceived agreement in term of R between SSM product values and in situ observation. To avoid seasonal effects, time series of anomalies from a moving monthly averaged are also computed. At each product grid and in situ station, the difference to the mean is calculated using a sliding window of five week and this difference is scaled by the standard deviation as in [30,57,58]. Anomaly SSM time series ($Ano\left(i\right)$, Equation (2) reflect the time-integrated impact of antecedent meteorological forcing. For each product SSM estimate as well as in situ observation at day i, a period P is defined with P = [i − 17, i + 17]. If at least five samples of SSM values are available in this period, the average SSM value ($\overline{SSM\left(P\right)}$) and the standard deviation ($stdev\left[SSM\left(P\right)\right])$ over each time windows are computed:

$$Ano\left(i\right)= \frac{SSM\left(i\right)-\overline{SSM\left(P\right)}}{stdev\left[SSM\left(P\right)\right]}$$

(2)

To summarize, bias, RMSD, and ubRMSD were computed using the original SSM values, while the R was computed using anomalies SSM values. Figure 2 shows the comparison between SSM products and in situ SSM measurements.

3.1. Using Together All the Data of All the Stations

For SMAP, results show that the three products tend to underestimate the in situ SSM values (Figure 2a–c and

Table 1. Statistics of the comparison between SSM products and in situ SSM measurements. The format of results is “a/b”, where “a” corresponds to statistics obtained using all measurements together of all stations, and “b” corresponds to station average statistics. For all products, p-value of the comparison was lower than 0.01 which indicates that the correlation is significant.

Products	Bias (vol. %)	RMSD (vol. %)	ubRMSD (vol. %)	R	N
SMAP-36 km	−5.5/−5.4	8.7/7.9	6.8/4.7	0.69/0.69	1847
SMAP-9 km	−3.5/−3.5	9.8/9.1	9.2/5.1	0.66/0.65	1793
SMAP/S1-1 km	−9.2/−8.5	10.6/10.0	5.4/4.3	0.48/0.48	259
ASCAT-25 km	−4.7/−3.7	9.1/7.8	7.9/6.2	0.50/0.49	5291
ASCAT-12.5 km	−5.2/−4.1	9.4/8.5	7.8/6.1	0.50/0.44	5252
ASCAT-1 km	−1.2/−1.3	7.5/7.4	7.4/6.2	0.55/0.56	942
SMOS-IC	−11.9/−10.9	14.8/13.3	8.7/5.3	0.56/0.57	1128
SMOS-NRT	−12.4/−11.2	14.5/12.7	7.6/5.6	0.43/0.47	900
SMOS-CATDS	−9.5/−9.6	12.3/12.1	7.8/6.9	0.38/0.42	1479
S1-25 km	−2.6/−2.2	6.7/6.2	6.1/5.6	0.49/0.60	459
S1-9 km	−3.7/−2.8	7.4/6.8	6.5/5.8	0.48/0.59	447
S1-1 km	−3.45/−2.6	7.2/6.8	6.3/5.6	0.50/0.59	462

). This underestimation is of −9.2 vol. % with SMAP/S1-1 km (L2_SSM) and about −4.5 vol. % with SMAP-36 km (L3_SSM) and SMAP-9 km (L3_SSM_E) products. The RMSD on SSM for the three SMAP products is close with 8.7 vol. % for SMAP-36 km, 9.8 vol. % for SMAP-9 km, and 10.6 vol. % for SMAP/S1-1 km. The ubRMSD for SMAP/S1-1 km (5.4 vol. %) is lower than that for SMAP-36 km and SMAP-9 km (6.8 vol. % and 9.2 vol. %, respectively). R is higher for SMAP-36 km and SMAP-9 km (~0.68) than for SMAP/S1-1 km (0.48).

Regarding ASCAT products (Figure 2d–f and Table 1), results show that ASCAT moderately underestimates the in situ soil moisture (bias about −4.9 vol. % for both ASCAT-25 km and ASCAT-12.5 km, and −1.2 vol. % for ASCAT-1 km). This is expected as the ASCAT soil moisture index was scaled using the in situ stations measurements to obtain soil moisture estimates (Section 2). RMSD and ubRMSD are respectively about 8.6 vol. % and about 7.7 vol. % for all ASCAT products. The R coefficient is the same for all ASCAT products (about 0.52).

Regarding SMOS products (Figure 2g–i and Table 1), results show that the three SMOS products underestimate the in situ soil moisture. This underestimation is slightly higher in the case of SMOS-IC and SMOS-NRT (bias about −12.1 vol. %) than in the case of SMOS-CATDS (bias of −9.5 vol. %). The RMSD, ubRMSD, and R are in the same order for the three SMOS products (RMSD about 13.9 vol. %, ubRMSD about 8.1 vol. %, and R about 0.46).

Finally, the comparison between the S1 products (spatial resolution of 25 km × 25 km, 9 km × 9 km, and 1 km × 1 km) and the in situ SSM measurements shows a bias about −3.2 vol. %, RMSD about 7.1 vol. %, ubRMSD about 6.3 vol. %, and R about 0.49 (Figure 2j–l and Table 1).

3.2. Using Each Station Alone and the Station Average of Statistics

Furthermore, the statistics (bias, RMSD, ubRMSD, and R) for each SSM product are computed using each station alone (Figure 3). S1 products show the lowest bias and RMSD with a bias between −6.0 vol. % and 2.6 vol. % and an RMSD between 2.8 vol. % and 8.5 vol. % (Figure 3g). The estimation of SSM from ASCAT products (bias between −10.5 and 5.1 vol. % and RMSD between 5.2 and 12.5 vol. %, Figure 3c), which are worse than those obtained with S1, are much better than those provided by SMOS and SMAP products. For instance, bias ranges between −13.6 and 11.9 vol. % and RMSD ranges between 3.9 and 15.0 vol. % with SMAP products (Figure 3a), and bias ranges between −22.5 and 4.0 vol. % and RMSD ranges between 5.3 and 23.5 vol. % with SMOS products (Figure 3e).

The ubRMSD values computed for each station alone are in the same order for the three SMAP products (ubRMSD between 3.3 and 6.9 vol. % for most of the stations, Figure 3b). R values show that for most of the stations SMAP/S1-1 km has a low correlation with in situ measurements (R between 0.09 and 0.63) whereas SMAP-36 km and SMAP-9 km products show high R-values (R between 0.50 and 0.85) (Figure 3b). For SMOS products, statistics computed using each station alone show that for most of the in situ stations the ubRMSD is in the same order with ubRMSD between 3.5 and 7.2 vol. % (Figure 3f). SMOS-IC and SMOS-NRT products are more correlated with the in situ measurements (R between 0.48 and 0.82) than SMOS-CATDS product (R between 0.26 and 0.39). Finally, statistics show that for most of the stations the range of R and ubRMSD is the same for ASCAT and S1 products with R between 0.27 and 0.93 and ubRMSD between 2.8 and 9.3 vol. % (Figure 3d,h).

For all SSM products, results show that the station average bias, which is the average of bias-values computed using each in situ station alone, is similar to bias obtained when using all in situ measurements of all stations together (absolute difference less than 1.2 vol. %) (Table 1). Similarly, the station average RMSD is close to the RMSD computed using all in situ measurements of all stations together (absolute difference less than 2 vol. %) (Table 1). However, the station average ubRMSD and R are different from ubRMSD and R obtained when using the measurements of all stations together (Table 1), which is expected as they are non-linear metrics. For all SMAP products, the station average ubRMSD is equal to 4.7 vol. % (Table 1). In addition, the station average R coefficient is about 0.67 for SMAP-36 km and SMAP-9 km and equal to 0.48 for SMAP/S1-1 km. All ASCAT products provide similar station average ubRMSD (about 6.2 vol. %) and R (about 0.49) station average. The station average ubRMSD is equal to 5.9 vol. % for all SMOS products and the station average R is equal to 0.57 for SMOS-IC and about 0.45 for SMOS-NRT and SMOS-CATDS. Finally, all S1 products have the same station average ubRMSD (about 5.6 vol. %) and R coefficient (about 0.59) (Table 1). The overall R and the station average R values differ more for S1 than for other products. This is because the number of samples used to evaluate each station is more important in the case of SMAP, ASCAT and SMOS products than in the case of S1 products. For S1 products the difference between the station average R and overall R values is about 0.1 and almost null for SMAP, ASCAT and SMOS products.

To investigate the effects of samples number on results, statistical values for ASCAT, SMOS, and SMAP (SMAP-36 km and SMAP-9 km) were also computed using the same number of samples as for S1 product (459 samples). For each station, data from SMAP, ASCAT, and SMOS were considered if their dates are within ± 2 days from S1 dates. Results show that for ASCAT, SMOS, and SMAP the use of these 459 samples only yields quite close statistical values as the use of all samples between 1 January 2016 and 30 Jun 2017. Indeed, the maximum change is of 1.2 vol. % for bias, 0.7 vol. % for RMSD, 1.0 vol. % for ubRMSD, and 0.04 for R. For instance, using all data between 1 January 2016 and 30 Jun 2017 the SMAP-36 km product has ubRMD of 4.7 vol. % and R of 0.69, while the use of same samples number as S1 (459) yields ubRMD of 4.4 vol. % and R of 0.75.

4. Discussion

Several statistical variables were used to evaluate SMAP, ASCAT, SMOS, and S1 SSM products with respect to in situ measurements. Station average bias and RMSD show that SMAP-36 km and SMAP-9 km products provide more precise soil moisture estimates than SMOS products (SMOS-IC, SMOS-NRT, and SMOS-CATDS). Station average bias obtained with SMAP-36 km and SMAP-9 km products is about −4.5 vol. % comparatively to about −10.5 vol. % obtained with SMOS products (SMOS-IC, SMOS-NRT, and SMOS-CATDS). This is in line with the findings of Al-Yaari et al. [59] who compared SMAP-L3 and SMOS-L3 over SMOSMANIA sites [59]. For the SMAP coarse resolution products, the station average RMSD is about 8.5 vol. % comparatively to about 12.7 vol. % obtained with SMOS products (SMOS-IC, SMOS-NRT, and SMOS-CATDS). The robustness of SMAP-36 km and SMAP-9 km products in comparison to the three SMOS products can be linked to the RFI mitigation which has more impact on SMOS than SMAP: SMAP has onboard spectral RFI filtering, where only ground segment mitigation is used for SMOS. Oliva et al. [12] showed that RFI caused by active sources emissions increases the brightness temperature of the area resulting in lower soil moisture estimates. Oliva et al. [12] reported that the SMOS SSM underestimates in situ SSM from the Valencia Anchor Station [60] (station designed to produce match ups for SMOS calibration and validation activities). The Valencia Anchor Station is located to the west of the city of Valencia, in Spain, which is an area very polluted with RFI mitigation. The underestimation was improved with the improvement of RFI situation [12]. Thus, the presence of RFI mitigation over our study area could explain the low accuracy of SMOS products in comparison to SMAP-36 km and SMAP-9 km products. In a study conducted in four watershed in USA, areas with low RFI, SMOS product (generated by SMOS algorithm v.400) was validated using in situ soil moisture networks and results showed that SMOS provides soil moisture estimates with a good accuracy (4.3 vol. %) [26]. Similarly, Al Bitar et al. [15] compared SMOS Level 2 User Data Products (SML2UDP) using in situ soil moisture networks that represents a variety of conditions across the USA where radio RFI in L-Band is very low and found that SMOS meets the mission requirement accuracy of 4 vol. %. Kerr et al. [61] evaluate the SMOS-CATDS and SMOS-NRT products through dense networks located in USA, Denmark, Mali, Niger and Benin and found that the these products achieve its expected goal in term of precision (Bias close to 0 and STDE “Standard Deviation of the Error” about 6 vol. % and R about 0.60). Finally, the station average ubRMSD show that SMAP and SMOS products have similar accuracy, and the station average R shows that SMAP and SMOS similarly capture the temporal dynamics of the in situ SSM.

Moreover, values of station average bias, station average RMSD, station average ubRMSD, and station average R show that that ASCAT and S1 provide soil moisture estimates with accuracy close to that obtained with SMAP coarse resolution products (SMAP-36 km and SMAP-9 km) (Table 1). For SMAP (SMAP-36 km and SMAP-9 km), ASCAT (ASCAT-25 km, ASCAT-12.5 km, and ASCAT-1 km), and S1 (S1-25 km, S1-9 km and S1-1 km) products the station average bias is about −3.2 vol. %, the station average RMSD is about 7.6 vol. %, the station average ubRMSD is about 5.6 vol. %, and the station average R is about 0.57. In a recent study, Kim et al. [43] evaluated the SMAP and ASCAT products using the SMOSMANIA network measurements between 2012 and 2015 (the same network used in this study) and found that SMAP and ASCAT data had similar accuracy (ubRMSD about 4 vol. % and R about 0.69).

In addition, Figure 2a,b,g–i show that the passive SSM products (SMAP and SMOS) provide estimated SSM values for the NBN station higher than those for the other stations. This could be explained by the fact that the SMAP and SMOS pixels that contain the NBN station are close to the sea, about 10 km away from the waterfront. For pixels near the coast, the sea water surfaces may contribute to the observed brightness temperature since the 3 dB sensor footprint has an original resolution of about 40 km × 40 km for SMAP and 50 km × 50 km for SMOS. The presence of a fraction of sea water surface within the SMAP or SMOS footprint lead to a decrease in the value of the observed brightness temperature and consequently to an overestimation of the ground truth SSM values. The SMAP-36 km NBN pixel (center: 43.33230°N, 2.80083°E) is strongly heterogeneous: it includes both crops in a lowland area and relatively dry coniferous forest vegetation (surface fraction ~38%) in a hilly area. The presence of this relatively dry area may the reason behind the lower retrieved SSM values for the SMAP-36 km NBN pixel, than for the SMAP-9 km NBN pixel (center: 43.18770°N, 2.94087°E), which includes mainly crops in the lowland area close to the waterfront.

Finally, results computed for each station alone show that the S1 products provide the most precise SSM estimates. The higher accuracy of estimated SSM moisture from S1 data could be due to (1) the use of well calibrated IEM [53,54,55] combined with the well parametrized WCM [52] to invert the S1 backscattering coefficients, (2) the use of a priori information defined in using precipitation records to improve the SSM estimation, and (3) the use of high resolution (10 m × 10 m) land cover map derived from S2 images to eliminate forest and urban pixels.

5. Conclusions

The aim of this study was to assess the accuracy of Surface Soil Moisture (SSM) at sites in southwestern France. Several SSM products obtained from four sensors were evaluated: SMAP (36 km × 36 km, 9 km × 9 km, and 1 km × 1 km), ASCAT (25 km × 25 km, 12.5 km × 12.5 km, and 1 km × 1 km), SMOS (SMOS-IC, SMOS-NRT, and SMOS-CATDS, 25 km × 25 km) and S1 (25 km × 25 km, 9 km × 9 km, and 1 km × 1 km). The accuracy of each product was computed using in situ measurements observed at a depth of 5 cm.

Results showed that:

(i)

the SMAP/S1-1 km has lower accuracy than SMAP-36 km and SMAP-9 km,

(ii)

the accuracy of SMAP-36 km and SMAP-9 km (station average bias about −4.5 vol. %, and station average RMSD about 8.5 vol. %) was better than that of SMOS products (SMOS-IC, SMOS-NRT, and SMOS-CATDS) (station average bias about −10.6 vol. %, and station average RMSD about 12.7 vol. %). On our study sites, this could be related to the presence of RFI noise that affects more SMOS brightness temperature measurements,

(iii)

the accuracy of SMAP products (SMAP-36 km and SMAP-9 km) was close to that of ASCAT (ASCAT-25 km, ASCAT-12.5 km and ASCAT-1 km) and S1 (S1-25 km, S1-9 km, and S1-1 km) products.

The station average statistics for SMAP, ASCAT and S1 SSM products were about −3.2 vol. % for the bias, 7.6 vol. % for the RMSD, 5.6 vol. % for the ubRMSD, and 0.57 for the R coefficient. When considering statistics computed for each station alone, one can conclude that S1 products gave the most precise soil moisture estimates.

It should be noted that the results obtained in this study cannot be generalized for other regions of the world. It is difficult to predict the performance of a SSM product over a region, without performing a quality assessment using in situ measurements on that region. Indeed, the performance of each product depends on many factors such as, but not limited to, soil type, climate, presence of noise (RFI in case of SMOS and SMAP) and land cover. Further studies should evaluate the performance of these products over other regions with different climate/vegetation conditions.