the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
04 Jan 2021
04 Jan 2021
Comprehensive evaluation of satellite-based and reanalysis soil moisture products using in situ observations over China
- 1School of Environment and Spatial Informatics, China University of Mining and Technology, Xuzhou, 221000, China
- 2Institute for Climate and Global Change Research, School of Atmospheric Sciences, Nanjing University, Nanjing, 210023, China
- 3Joint International Research Laboratory of Atmospheric and Earth System Sciences, Nanjing University, 210023, Nanjing, China
- 4Department of Remote Sensing, Helmholtz Centre for Environmental Research−UFZ, Permoserstrasse 15, 04318, Leipzig, Germany
- 5Remote Sensing Centre for Earth System Research, Leipzig University, 04103, Leipzig, Germany
- 1School of Environment and Spatial Informatics, China University of Mining and Technology, Xuzhou, 221000, China
- 2Institute for Climate and Global Change Research, School of Atmospheric Sciences, Nanjing University, Nanjing, 210023, China
- 3Joint International Research Laboratory of Atmospheric and Earth System Sciences, Nanjing University, 210023, Nanjing, China
- 4Department of Remote Sensing, Helmholtz Centre for Environmental Research−UFZ, Permoserstrasse 15, 04318, Leipzig, Germany
- 5Remote Sensing Centre for Earth System Research, Leipzig University, 04103, Leipzig, Germany
Abstract. Soil moisture (SM) plays a critical role in the water and energy cycles of the earth system; consequently, a long-term SM product with high quality is urgently needed. In this study, five SM products, including one microwave remote sensing product [European Space Agency's Climate Change Initiative (ESA CCI)] and four reanalysis datasets [European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis-Interim (ERAI), National Centers for Environmental Prediction (NCEP), the Twentieth Century Reanalysis Project from National Oceanic and Atmospheric Administration (NOAA) and the European Centre for Medium-Range Weather Forecasts Reanalysis 5 (ERA5)], are systematically evaluated using in situ measurements during 1981–2013 in four climate regions at different timescales over mainland China. The results show that ESA CCI is closest to the observations in terms of both the spatial distributions and magnitude of the monthly SM. All reanalysis products tend to overestimate soil moisture in all regions but have higher correlations than the remote sensing product except in Northwest China. The largest inconsistency is found in southern Northeast China, with a relative RMSE value larger than 0.1. However, none of the products can well reproduce the trends of interannual anomalies. The largest relative bias of 44.6 % is found for the ERAI SM product under severe drought conditions, and the lowest relative biases of 4.7 % and 9.5 % are found for the ESA CCI SM product under severe drought conditions and the NCEP SM product under normal conditions, respectively. As decomposing mean square errors in all the products suggests that the bias terms are the dominant contribution, the ESA CCI SM product is a good option for long-term hydrometeorological applications in mainland China. ERA5 is also a promising product, which is attributed to the incorporation of more observations. This long-term intercomparison study provides clues for SM product enhancement and further hydrological applications.
Xiaolu Ling et al.
Status: closed
-
RC1: 'Comment on hess-2020-611', Anonymous Referee #1, 12 Jan 2021
Soil moisture, as one of the essential climate variables, has attracted more and more attention from climate research. However, there is still a long way to go for the recently widely used soil moisture products, including reanalyses based on models and retrievals from remotely sensed data, to be comparable with observations. To further develop and properly use them, it is necessary to compare with in situ observations to reveal their uncertainties. In this manuscript, the five satellite-based and reanalysis soil moisture products were evaluated in China with in situ observations for top soil layer (0-10 cm). By now the manuscript still needs to further discuss the uncertainties of in situ observations of soil moisture data, the influence of sparse data samples, and thus the unfair to compare grid products using point-scale measurements. In particular, the author pointed out that the bias term controlled the deviations of soil moisture products from the observed values. This partly stems from the spatial mismatches in the comparisons of the soil moisture measured at a point with model grid means. So, it requires more discussion about its implications. In addition, the method part needs to provide more details, for example, how the monthly means were estimated using 3-sample observations per month.
Specific comments and suggestions:
- “mainland China” is NOT a right term, you can use: the Chinese Mainland, Mainland of China or China’s Mainland.
- 2.1.1 ESA CCI SM, how the various retrievals of the passive and active sensors combined should be detailed a bit more, for example, using land surface model products?
- 2.1.5 ERA5 SM, the improvements of land processes in ERA5 against ERAI are helpful to understanding of the results with respect to in situ soil moisture in these two reanalses.
- 2.2 In Situ SM and Preprocessing of Datasets, the in situ observations were took from three datasets, so details about difference in the operation of measurements and the means of quality control for the datasets are necessary to assess the credibility of in situ data.
- The ‘CN05.1’ should be defined before its first citation.
- Line 155, ‘different drought/well conditions’, ‘well’ is a typing error?
- More detailed information on the decomposition of MSEs and the test methods is necessary for potential readers.
- Fig. 2, the spatial pattern for ERA-Interim looks pretty different from that for ERA5 and others, especially across the arid northwest and regions along the coasts. Please doublecheck it, otherwise, give an explanation.
- Line 195, the larger rRMSEs in the Yangtze-Huai basin may be associated with the irrigation influence on the in situ observations. However, it’s hard to think of its direct links to monsoon precipitation.
- Fig. 4, the regionally averaged observations show higher soil moisture in NW than the other three regions. It is NOT consistent with the precipitation patterns in Fig. 1. The discrepancy should be discussed a little bit more.
- 3.2.2 Seasonality, since the previous results talk about the summer (JJA) soil moisture comparisons with observations, how the seasonal soil moisture were selected in this section should be clarified further. Further, the soil moisture discussed in the manuscript focused on the top soil layer (0-10 cm), so I guess its seasonality connected closely to precipitation annual cycle. However, in Fig. 6, it looks not so, please discuss it further.
- Line 230, ‘snow or frozen soil during these periods.’ The frozen seasons should be excluded in the comparisons, otherwise the model soil moisture is virtually a different variable from the observed.
- Line 286, ‘The SCâPDSI is utilized (Wells et al., 2004).’, for what is SC-PDSI used?
-
AC1: 'Reply on RC1', Xiaolu Ling, 30 Mar 2021
The comment was uploaded in the form of a supplement: https://hess.copernicus.org/preprints/hess-2020-611/hess-2020-611-AC1-supplement.pdf
-
RC2: 'Comment on hess-2020-611', Anonymous Referee #2, 21 Jan 2021
This interesting analysis used in-situ observations in China to evaluate several reanalysis- and RS-based SM products. While it is a nice self-contained study with seemingly comprehensive analyses, I found the study lacking sufficient physical explanations supporting several findings of their analyses. Also, some figures are not very well presented and need to be updated. Therefore, I’d suggest the authors go through moderate revisions before this paper can be publishable. Below are some suggestions to improve the paper:
Insufficient explanations/supports:
- 6: ESA-CCI seems to not represent seasonality well. Why? It seems no variation there. I think this explanation on “which may be because of snow or frozen soil during these periods” is too thin. To me this still does not explain well on why worst seasonality are there.
- 8: it seems discouraging that none of the products available captures the anomalies well especially in NC. Can the author provide some feasible explanations on why this is the case, and discuss how this could influence applications in those regions and what are the potential future directions for improvements?
- Line 301: I think “which is partly due to the combined influence of longwave and shortwave radiation” does not sufficiently explain why low correlation there. Please expand what you mean exactly. Also, if separation of LW and SW radiation helps, would it be possible to use LW and SW data to re-draw this scatter plot?
- 12 & L298-L302: overall I think it’s an interesting figure. However, authors fail to explain in more detail on the underlying physical mechanisms responsible for these correlations and why they wanted to perform these analyses. This paragraph is too thin. In addition, It seems these plots are more driven by the availability of data, instead of driven by hypothesis testing needs. It would be helpful for the authors to put more thoughts on this figure and provide readers with more insights on why they chose to do the analysis and what’s new after doing the analysis.
Figure presentation problems:
- It is very difficult to distinguish in-situ line in Fig. 6 as it can be confused with ERA-5. I’d suggest to use thicker black line to denote in-situ observations in Fig. 6. Also, be better to use consistent legend with Fig. 4 & Fig. 8.
- 7: I think it would be very difficult for readers to directly extract useful information from this figure, partly because of the color bar used, which makes it all red (plus there are so many panels). I’d suggest to use more continuous colors, with more contrasting from 0-1, such that differences in the correlations are better presented. Since only very few locations show negative correlations, you can cap the lower bound at 0, and just mention “limited negative correlation” in the caption. This way, 0-1 can be better contrasted (using blue to red) to support your interpretation on the figure in the main text.
- 10 & Fig. 11: the caption is incomplete and misleading. It did not mention which skill metrics is plotted here. Please mention it explicitly in the caption. Also, draw a reference line on 0 such that readers know where to expect good performance.
Minor:
- I do not think the literature review is comprehensive. Beck et al. (2021; HESS) presented a much more comprehensive study performed at the global scale. It should be included in the Introduction and discussions on relevance to your study needs to be mentioned. I disagree with the claim in L62 that no long-term SM products have been compared with ESA-CCI. Please revise accordingly.
Overall comment:
- In fact, I like the study very well because it is self-contained, with comprehensive analysis, and the writing is good too. However, I am thinking what could be more useful to the community, is perhaps for the authors to share their in-situ soil moisture observations through posting the data via figshare or other publicly accessible data portal. It seems to me that this study is only unique because of its observations, which are generally not shared with the public. If the data can be shared properly with the whole community, people may find more innovative ways of using the data for other research purposes such as drought monitoring. Is this something that the authors are considering? It could be helpful to at least comment on or discuss this issue in an academic paper.
-
AC2: 'Reply on RC2', Xiaolu Ling, 30 Mar 2021
The comment was uploaded in the form of a supplement: https://hess.copernicus.org/preprints/hess-2020-611/hess-2020-611-AC2-supplement.pdf
-
CC1: 'Comment on hess-2020-611', Xingwang Fan, 21 Jan 2021
This is an important study regarding the evaluation of long-term soil moisture products within mainland China. Several products, both reanalyzed and remotely sensed, were comprehensively evaluated in the spatial and temporal domain, and under hydrologically dry, normal and wet scenarios. The driving forces of soil moisture dynamics were examined with respect to precipitation, temperature and radiation (shortwave plus longwave). In my opinion, however, the major concern is spatial mismatch (both horizontal and vertical) between various products and in-situ measurements. In addition, most of current explanations and reasonings lack a solid physical background.
- The authors need to reappraise their motive of this study, because NOAA and NCEP soil moisture (SM) products (a spatial resolution of 2 degrees) are usually not qualified for hydro-meteorological studies (flood or drought as reviewed by Peng et al. 2020, in Remote Sensing of Environment) in mainland China. As pointed out by the other reviewer, such coarse spatial resolutions would cause errors of representativeness. Although spatial averaging to some extent can alleviate such an effect, I still think errors of representativeness (together with differences in effective soil depth) might contribute substantially to the bias values. That is probably the reason why CCI (a spatial resolution of 0.25 degrees) and ERA-5 (a spatial resolution of 31 kilometers) have a slightly better performance.
- The presentation of results should be improved. In numerous cases, the authors repeat the overestimation of modelling SM data and the underestimation of remotely sensed SM data.
- Some descriptions contradict each other throughout the manuscript. For example, in Lines 88-90, the authors first report the underestimation in northwest China and then report the opposite side.
- Line 79, the authors promise to discuss on sources of SM errors. However, most of the explanations are speculations and even key words. In Line 223 for example, why different land surface types and varying soil parameters cause differences between CCI and model outputs? In Line 227, how vegetation presence leads to a clear SM seasonal cycle? In Line 237, how precipitation and frozen soils increase autocorrelation? Then in the following sentence, what particular soil type and texture decreases autocorrelation?
- Section 3.2.2, how autocorrelation is related to the performance of soil moisture products.
- Lines 288-289, is this a manifestation of scaling effect? Spatial averaging (coarse resolution) masks out extremely low and high SM values.
- Line 45, “temporarily” should be “temporally”.
- Line 65-66, this sentence makes no sense.
- Line 86, “plus” is incorrect here.
- Line 87, delete “underlying”.
- The method section should provide more details, such as data interpolation in the vertical direction. The CCI has a penetration depth of < 2 cm, and the effective soil depth for model outputs is 0-10 cm, and the in-situ measurement depth is 10 cm. Such differences might also cause errors of representativeness.
- Line 166, “climate” should be “climatological”.
- Line 178, “Discussions” should be “Discussion”.
- Lines 184-185, this sentence has been already in the previous section, and it does not belong to Result section.
- Line 199, improper use of “According to”.
- Line 214, what kind of mechanism?
- Line 217, what is “variability performance”?
- Lines 17-18, this sentence “demonstrating…” makes no sense.
- Line 232, the snow-covered and frozen grids were not removed in this study?
- Line 300, the explanations are unclear and confusing.
- Line 321, it is not quite right to say “CCI is not useful”.
- Why not use GLDAS (the same grid resolution as CCI) or CLDAS (more spatial details) data as validation reference? Although with a shorter temporal coverage, other optimized SM data in mainland China can also serve as references. Using these data reduces errors of representativeness. Perhaps shorter time series also works assuming a temporally stable data quality of various SM products.
-
AC3: 'Reply on CC1', Xiaolu Ling, 30 Mar 2021
The comment was uploaded in the form of a supplement: https://hess.copernicus.org/preprints/hess-2020-611/hess-2020-611-AC3-supplement.pdf
Status: closed
-
RC1: 'Comment on hess-2020-611', Anonymous Referee #1, 12 Jan 2021
Soil moisture, as one of the essential climate variables, has attracted more and more attention from climate research. However, there is still a long way to go for the recently widely used soil moisture products, including reanalyses based on models and retrievals from remotely sensed data, to be comparable with observations. To further develop and properly use them, it is necessary to compare with in situ observations to reveal their uncertainties. In this manuscript, the five satellite-based and reanalysis soil moisture products were evaluated in China with in situ observations for top soil layer (0-10 cm). By now the manuscript still needs to further discuss the uncertainties of in situ observations of soil moisture data, the influence of sparse data samples, and thus the unfair to compare grid products using point-scale measurements. In particular, the author pointed out that the bias term controlled the deviations of soil moisture products from the observed values. This partly stems from the spatial mismatches in the comparisons of the soil moisture measured at a point with model grid means. So, it requires more discussion about its implications. In addition, the method part needs to provide more details, for example, how the monthly means were estimated using 3-sample observations per month.
Specific comments and suggestions:
- “mainland China” is NOT a right term, you can use: the Chinese Mainland, Mainland of China or China’s Mainland.
- 2.1.1 ESA CCI SM, how the various retrievals of the passive and active sensors combined should be detailed a bit more, for example, using land surface model products?
- 2.1.5 ERA5 SM, the improvements of land processes in ERA5 against ERAI are helpful to understanding of the results with respect to in situ soil moisture in these two reanalses.
- 2.2 In Situ SM and Preprocessing of Datasets, the in situ observations were took from three datasets, so details about difference in the operation of measurements and the means of quality control for the datasets are necessary to assess the credibility of in situ data.
- The ‘CN05.1’ should be defined before its first citation.
- Line 155, ‘different drought/well conditions’, ‘well’ is a typing error?
- More detailed information on the decomposition of MSEs and the test methods is necessary for potential readers.
- Fig. 2, the spatial pattern for ERA-Interim looks pretty different from that for ERA5 and others, especially across the arid northwest and regions along the coasts. Please doublecheck it, otherwise, give an explanation.
- Line 195, the larger rRMSEs in the Yangtze-Huai basin may be associated with the irrigation influence on the in situ observations. However, it’s hard to think of its direct links to monsoon precipitation.
- Fig. 4, the regionally averaged observations show higher soil moisture in NW than the other three regions. It is NOT consistent with the precipitation patterns in Fig. 1. The discrepancy should be discussed a little bit more.
- 3.2.2 Seasonality, since the previous results talk about the summer (JJA) soil moisture comparisons with observations, how the seasonal soil moisture were selected in this section should be clarified further. Further, the soil moisture discussed in the manuscript focused on the top soil layer (0-10 cm), so I guess its seasonality connected closely to precipitation annual cycle. However, in Fig. 6, it looks not so, please discuss it further.
- Line 230, ‘snow or frozen soil during these periods.’ The frozen seasons should be excluded in the comparisons, otherwise the model soil moisture is virtually a different variable from the observed.
- Line 286, ‘The SCâPDSI is utilized (Wells et al., 2004).’, for what is SC-PDSI used?
-
AC1: 'Reply on RC1', Xiaolu Ling, 30 Mar 2021
The comment was uploaded in the form of a supplement: https://hess.copernicus.org/preprints/hess-2020-611/hess-2020-611-AC1-supplement.pdf
-
RC2: 'Comment on hess-2020-611', Anonymous Referee #2, 21 Jan 2021
This interesting analysis used in-situ observations in China to evaluate several reanalysis- and RS-based SM products. While it is a nice self-contained study with seemingly comprehensive analyses, I found the study lacking sufficient physical explanations supporting several findings of their analyses. Also, some figures are not very well presented and need to be updated. Therefore, I’d suggest the authors go through moderate revisions before this paper can be publishable. Below are some suggestions to improve the paper:
Insufficient explanations/supports:
- 6: ESA-CCI seems to not represent seasonality well. Why? It seems no variation there. I think this explanation on “which may be because of snow or frozen soil during these periods” is too thin. To me this still does not explain well on why worst seasonality are there.
- 8: it seems discouraging that none of the products available captures the anomalies well especially in NC. Can the author provide some feasible explanations on why this is the case, and discuss how this could influence applications in those regions and what are the potential future directions for improvements?
- Line 301: I think “which is partly due to the combined influence of longwave and shortwave radiation” does not sufficiently explain why low correlation there. Please expand what you mean exactly. Also, if separation of LW and SW radiation helps, would it be possible to use LW and SW data to re-draw this scatter plot?
- 12 & L298-L302: overall I think it’s an interesting figure. However, authors fail to explain in more detail on the underlying physical mechanisms responsible for these correlations and why they wanted to perform these analyses. This paragraph is too thin. In addition, It seems these plots are more driven by the availability of data, instead of driven by hypothesis testing needs. It would be helpful for the authors to put more thoughts on this figure and provide readers with more insights on why they chose to do the analysis and what’s new after doing the analysis.
Figure presentation problems:
- It is very difficult to distinguish in-situ line in Fig. 6 as it can be confused with ERA-5. I’d suggest to use thicker black line to denote in-situ observations in Fig. 6. Also, be better to use consistent legend with Fig. 4 & Fig. 8.
- 7: I think it would be very difficult for readers to directly extract useful information from this figure, partly because of the color bar used, which makes it all red (plus there are so many panels). I’d suggest to use more continuous colors, with more contrasting from 0-1, such that differences in the correlations are better presented. Since only very few locations show negative correlations, you can cap the lower bound at 0, and just mention “limited negative correlation” in the caption. This way, 0-1 can be better contrasted (using blue to red) to support your interpretation on the figure in the main text.
- 10 & Fig. 11: the caption is incomplete and misleading. It did not mention which skill metrics is plotted here. Please mention it explicitly in the caption. Also, draw a reference line on 0 such that readers know where to expect good performance.
Minor:
- I do not think the literature review is comprehensive. Beck et al. (2021; HESS) presented a much more comprehensive study performed at the global scale. It should be included in the Introduction and discussions on relevance to your study needs to be mentioned. I disagree with the claim in L62 that no long-term SM products have been compared with ESA-CCI. Please revise accordingly.
Overall comment:
- In fact, I like the study very well because it is self-contained, with comprehensive analysis, and the writing is good too. However, I am thinking what could be more useful to the community, is perhaps for the authors to share their in-situ soil moisture observations through posting the data via figshare or other publicly accessible data portal. It seems to me that this study is only unique because of its observations, which are generally not shared with the public. If the data can be shared properly with the whole community, people may find more innovative ways of using the data for other research purposes such as drought monitoring. Is this something that the authors are considering? It could be helpful to at least comment on or discuss this issue in an academic paper.
-
AC2: 'Reply on RC2', Xiaolu Ling, 30 Mar 2021
The comment was uploaded in the form of a supplement: https://hess.copernicus.org/preprints/hess-2020-611/hess-2020-611-AC2-supplement.pdf
-
CC1: 'Comment on hess-2020-611', Xingwang Fan, 21 Jan 2021
This is an important study regarding the evaluation of long-term soil moisture products within mainland China. Several products, both reanalyzed and remotely sensed, were comprehensively evaluated in the spatial and temporal domain, and under hydrologically dry, normal and wet scenarios. The driving forces of soil moisture dynamics were examined with respect to precipitation, temperature and radiation (shortwave plus longwave). In my opinion, however, the major concern is spatial mismatch (both horizontal and vertical) between various products and in-situ measurements. In addition, most of current explanations and reasonings lack a solid physical background.
- The authors need to reappraise their motive of this study, because NOAA and NCEP soil moisture (SM) products (a spatial resolution of 2 degrees) are usually not qualified for hydro-meteorological studies (flood or drought as reviewed by Peng et al. 2020, in Remote Sensing of Environment) in mainland China. As pointed out by the other reviewer, such coarse spatial resolutions would cause errors of representativeness. Although spatial averaging to some extent can alleviate such an effect, I still think errors of representativeness (together with differences in effective soil depth) might contribute substantially to the bias values. That is probably the reason why CCI (a spatial resolution of 0.25 degrees) and ERA-5 (a spatial resolution of 31 kilometers) have a slightly better performance.
- The presentation of results should be improved. In numerous cases, the authors repeat the overestimation of modelling SM data and the underestimation of remotely sensed SM data.
- Some descriptions contradict each other throughout the manuscript. For example, in Lines 88-90, the authors first report the underestimation in northwest China and then report the opposite side.
- Line 79, the authors promise to discuss on sources of SM errors. However, most of the explanations are speculations and even key words. In Line 223 for example, why different land surface types and varying soil parameters cause differences between CCI and model outputs? In Line 227, how vegetation presence leads to a clear SM seasonal cycle? In Line 237, how precipitation and frozen soils increase autocorrelation? Then in the following sentence, what particular soil type and texture decreases autocorrelation?
- Section 3.2.2, how autocorrelation is related to the performance of soil moisture products.
- Lines 288-289, is this a manifestation of scaling effect? Spatial averaging (coarse resolution) masks out extremely low and high SM values.
- Line 45, “temporarily” should be “temporally”.
- Line 65-66, this sentence makes no sense.
- Line 86, “plus” is incorrect here.
- Line 87, delete “underlying”.
- The method section should provide more details, such as data interpolation in the vertical direction. The CCI has a penetration depth of < 2 cm, and the effective soil depth for model outputs is 0-10 cm, and the in-situ measurement depth is 10 cm. Such differences might also cause errors of representativeness.
- Line 166, “climate” should be “climatological”.
- Line 178, “Discussions” should be “Discussion”.
- Lines 184-185, this sentence has been already in the previous section, and it does not belong to Result section.
- Line 199, improper use of “According to”.
- Line 214, what kind of mechanism?
- Line 217, what is “variability performance”?
- Lines 17-18, this sentence “demonstrating…” makes no sense.
- Line 232, the snow-covered and frozen grids were not removed in this study?
- Line 300, the explanations are unclear and confusing.
- Line 321, it is not quite right to say “CCI is not useful”.
- Why not use GLDAS (the same grid resolution as CCI) or CLDAS (more spatial details) data as validation reference? Although with a shorter temporal coverage, other optimized SM data in mainland China can also serve as references. Using these data reduces errors of representativeness. Perhaps shorter time series also works assuming a temporally stable data quality of various SM products.
-
AC3: 'Reply on CC1', Xiaolu Ling, 30 Mar 2021
The comment was uploaded in the form of a supplement: https://hess.copernicus.org/preprints/hess-2020-611/hess-2020-611-AC3-supplement.pdf
Xiaolu Ling et al.
Xiaolu Ling et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 433 | 174 | 17 | 624 | 2 | 3 |
- HTML: 433
- PDF: 174
- XML: 17
- Total: 624
- BibTeX: 2
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1