the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 09 Oct 2023
Evaluation of reanalysis soil moisture products using Cosmic Ray Neutron Sensor observations across the globe
Abstract. Accurate soil moisture information is vital for flood and drought predictions, crop growth and agricultural water management. Reanalysis soil moisture products with multi-decadal temporal coverage are gradually becoming a good alternative for providing global soil moisture data in various applications compared to in-situ measurements and satellite products. Much effort has been devoted to evaluating the performance of soil moisture products, yet the scale discrepancy between point measurements and grid cell soil moisture products limits the assessment quality. As the land surface and hydrological modelling community evolve towards the next generation of (sub)kilometer resolution models, Cosmic Ray Neutron Sensors (CRNS) that provide estimates of root-zone soil moisture at the field scale (~250 m radius from the sensor and up to 0.7 m deep), may consequently be more suitable for soil moisture product evaluation as they cover a relatively larger footprint, when compared to traditional methods. In this study, we perform a comprehensive evaluation of seven widely-used reanalysis soil moisture products (ERA5-Land, CFSv2, MERRA2, JRA55, GLDAS-Noah, CRA40 and GLEAM datasets) against 135 CRNS sites from the UK, Europe, USA and Australia. We evaluate the products using six metrics capturing different aspects of soil moisture dynamics. Results show that all reanalysis products exhibit good temporal correlation with the measurements, with the median of temporal correlation coefficient (R) values spanning from 0.69 to 0.79, though large deviations are found at sites with seasonally varying vegetation cover. Poor performance is observed across products for soil moisture anomalies timeseries, with R values varying from 0.49 to 0.70. The performance of reanalysis products differs greatly across regions, climate, land covers and topographic conditions. In general, all products tend to overestimate in arid climates and underestimate in humid regions as well as grassland. Most reanalysis products perform poorly in steep terrain. Relatively low temporal correlation and high Bias are detected in some sites from west of the UK, which might be associated with relatively low bulk density and high soil organic carbon. Overall, ERA5-Land, CFSv2, CRA40, GLEAM exhibit superior performance compared to MERRA2, GLDAS-Noah and JRA55. We recommend ERA5-Land and CFSv2 should be used in humid climates, whereas CRA40 and GLEAM perform better in arid regions. GLEAM is more effective in shrubland regions. Our findings also provide insights on directions for improvement of soil moisture products for product developers.
- Preprint
(1946 KB) - Metadata XML
-
Supplement
(1373 KB) - BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on hess-2023-224', Anonymous Referee #1, 15 Nov 2023
The authors present a noteworthy study that compares CRNS stations across multiple continents with remote sensing/reanalysis soil moisture products. Utilizing the CRSPY tool, the authors reprocess the raw CRNS data, addressing gaps in information, such as the atmospheric moisture correction (absent in the COSMOS USA network), and harmonize the dataset. Undoubtedly, this tool holds promise for advancing the CRNS method and enhancing the global CRNS community. The article is well-crafted, with comprehensive data documentation in both the appendix and online. However, I have some suggestions and comments that should be addressed before publication.
Major Suggestion:
I find some ambiguity regarding the reprocessing of CRNS data with CRSPY, particularly in the computation of the N0 parameter. The main article and supplemental material lack any mention of N0. To calculate N0, each site typically requires at least one gravimetric calibration survey, composed of 12-18 soil profiles within the footprint, sampled every 5 cm down to 30 cm. The gravimetric survey data is then weighted (following Kohli 2015/2017, etc.), and the Desilets 2010 function is inverted to determine N0. Most CRNS sites have gravimetric calibration data for one or more sample periods. Please include a detailed description of the process for estimating N0 at each site, specifying the gravimetric calibration dataset used (single or average of multiple calibrations, etc.). The accuracy of the N0 estimate is crucial, as incomplete or non-representative data may introduce significant bias in the comparison with reanalysis products. Best practices for N0 involve 2-3 calibration dates (Iwema 2015), though this is labor-intensive and not consistently implemented.
I am a bit confused by the description of soil organic carbon. Does this include both the organic carbon and mineral lattice water values? The variation in lattice water was found to be important for many of the original CRNS networks (Zreda 2012 COSMOS, Hawdon 2014 COSMOZ, etc.). The paper and appendix does not include the description of lattice water and should be clarified or added to the metadata.
Comment: The influence of rapidly growing crops on the CRNS observations remains a challenge. There have been several attempts to provide correction factors (either on N0 itself or on the moderated counts) but nothing definitive has been adopted by the community. The authors mention this in the limitations. I hope the community comes to a consensus soon about how best to deal with CRNS data in croplands. The influence of forest biomass seems to be an even more challenging problem but equally important.
Minor comment:
Figure 2. Label the 4 subplots a-d and list what geographical region they are from. This was not clear from the description.
Citation: https://doi.org/10.5194/hess-2023-224-RC1 -
AC1: 'Reply on RC1', Yanchen Zheng, 20 Dec 2023
Reply to Reviewer #1 Comments
Author responses are in bold and italic.The authors present a noteworthy study that compares CRNS stations across multiple continents with remote sensing/reanalysis soil moisture products. Utilizing the CRSPY tool, the authors reprocess the raw CRNS data, addressing gaps in information, such as the atmospheric moisture correction (absent in the COSMOS USA network), and harmonize the dataset. Undoubtedly, this tool holds promise for advancing the CRNS method and enhancing the global CRNS community. The article is well-crafted, with comprehensive data documentation in both the appendix and online. However, I have some suggestions and comments that should be addressed before publication.
Thanks for your valuable and positive comments of our paper. We appreciate your insightful comments, and we are committed to addressing the mentioned points in our revision.
Major Suggestion:
I find some ambiguity regarding the reprocessing of CRNS data with CRSPY, particularly in the computation of the N0 parameter. The main article and supplemental material lack any mention of N0. To calculate N0, each site typically requires at least one gravimetric calibration survey, composed of 12-18 soil profiles within the footprint, sampled every 5 cm down to 30 cm. The gravimetric survey data is then weighted (following Kohli 2015/2017, etc.), and the Desilets 2010 function is inverted to determine N0. Most CRNS sites have gravimetric calibration data for one or more sample periods. Please include a detailed description of the process for estimating N0 at each site, specifying the gravimetric calibration dataset used (single or average of multiple calibrations, etc.). The accuracy of the N0 estimate is crucial, as incomplete or non-representative data may introduce significant bias in the comparison with reanalysis products. Best practices for N0 involve 2-3 calibration dates (Iwema 2015), though this is labor-intensive and not consistently implemented.
I am a bit confused by the description of soil organic carbon. Does this include both the organic carbon and mineral lattice water values? The variation in lattice water was found to be important for many of the original CRNS networks (Zreda 2012 COSMOS, Hawdon 2014 COSMOZ, etc.). The paper and appendix does not include the description of lattice water and should be clarified or added to the metadata.We agree that in the current paper we didn’t mention about the information of N0 and lattice water for CRNS sites. We will add descriptions of these in the paper and also include the N0 and lattice water data for the available sites in the supplementary file.
Comment: The influence of rapidly growing crops on the CRNS observations remains a challenge. There have been several attempts to provide correction factors (either on N0 itself or on the moderated counts) but nothing definitive has been adopted by the community. The authors mention this in the limitations. I hope the community comes to a consensus soon about how best to deal with CRNS data in croplands. The influence of forest biomass seems to be an even more challenging problem but equally important.
As for the influence of vegetation effects on CRNS observations, thanks for agreeing to this point. We share the hope that the research community will collectively work towards developing more effective solutions to address and mitigate these effects in future studies.
Minor comment:
Figure 2. Label the 4 subplots a-d and list what geographical region they are from. This was not clear from the description.Thanks for spotting the missing labels of Figure 2, we will make sure to add the geographical region and a-d label in Figure 2 in our revision.
Citation: https://doi.org/10.5194/hess-2023-224-AC1
-
AC1: 'Reply on RC1', Yanchen Zheng, 20 Dec 2023
-
RC2: 'Comment on hess-2023-224', Anonymous Referee #2, 20 Nov 2023
The paper analyses the performance of global reanalysis soil moisture products against soil moisture obtained from cosmic ray neutron sensors. The results are analysed using different performance criteria, region specific and climate specific. The paper addresses highly relevent scientific questions, is exceptionally well structured, provides novel insights thereby presenting novel concepts and setting the global data sets into new context. The conclusions are reached, to the point and substantial.
However, there are two main improvements to be made before acceptance.
First: I suggest the authors consider to include SMAP Level 4 data as reanalysis product for evaluation. SMAP Level 4 comes as data assimilation product and is considered state-of-the-art soil moisture for the recent years. Without SMAP Level 4, the study seems incomplete.
Second: the terminology in Figure 10 as "highly recommended", "recommended" and "not recommended" must be adapted and if needed must be adapted throughout the text. All reanalysis products show high bias, and hence, the ERA5-Land cannot be highly recommended for humid sites as the authors have done in Figure 10. It should rather be recommended with care. I provide a suggestion below.
Minor comments:
Line 11: It is recommended to shorten the abstract. Lines 11-19 can be reduced to 3 lines. This will focus the reader on the methods, novelty, and key findings. These are mentioned in the abstract.Line 21: Although UK has its own network, it reads awkwards to read "sites from UK, Europe, USA and Australia" as a list because the UK is included in Europe and should not be listed beside Europe as separate geogrphic entity. Please not that this list refers to CRNS networks COSMOS-UK, COSMOS-Europe...
Line 42: Please also list this newer review on reanalysis, were a definition of reanalysis products is given https://doi.org/10.1029/2020RG000715
Line 50: This is well formulated. However, SMAP Level 4 is missing in this list https://doi.org/10.1029/2019MS001729
Line 115: rephrase to "harmonized processing of CRNS datasets".
Line 132: Where is SMAP Level 4? Please give strong reason for not using the most recent Soil Moisture Product https://doi.org/10.1029/2019MS001729 SMAP Level 4 is a reanalysis product as it includes meteorological variables, and satellite observations that are used to update soil moisture in a data assimilation framework.
In fact, I strongly suggest to add SMAP Level 4. As such, the study is out-dated. Adding SMAP Level 4 should be a short exercise - multiplying the scientific impact of this study by a factor. Hopefully SMAP Level 4 performs best amongst all Reanalysis products. Although it is not 20 years length, it is the SM reanalysis product that will be used for recent years rather than any of the other products analyzed. The technical definition of 20+ years (line 42) by a reference of 2005 is not sufficient to not use the most up-to-date SM reanalysis product in this highly relevant global study.
Line 164: This depends on the source of uncertainty. Daily averaging causes loss of signal and proper filtering maintains signal while reducing uncertainty/noise. High measurement uncertainty can be compensated for by applying temporal filtering methods or simple daily averaging e.g. https://doi.org/10.3390/s22239143 .
Line 174: Rephrase towards more active voice: "Spatial scale matching"
Line 184: see comment before.
Figure 2: Add region names and a/b/c/d to the figure ( or names of the networks ).
Line 290: Clarify meaning of "Figure 4cd". And typically it is called "US sites" rather than "USA sites".
Line 306: Clarify the use of "main paper" which implies, there is a secondary paper.
Line 318: "low temporal correlation" directly contradicts the abstract line 22: "all reanalysis products exhibit good temporal correlation with
the measurements". In the abstract, I suggest to add "products generally exhibit" to weaken this statement in the abstract.Line 325: Please rephrase. Sites cannot be a reason for low performance. Reasons for low performance can only be process related or of technical nature. A site itself cannot be the reason for low performance.
Line 329: CFSv2 or CFSRv2?
Line 330: I contradict, a model cannot perform best in all statistical metrics except for bias. High bias produces high MSE. With a poor bias, the MSE should be poor as well. Please clarify.Line 345: better compared to what? Please specify.
Line 383: Please add that vegetation correction were proposed e.g. https://doi.org/10.1002/2014WR016443
Line 385: Please discuss in a sentence your study results with these results: https://doi.org/10.5194/hess-21-6201-2017
Line 407: lower performance is preferred to "worse" performance which comes with a very negative connotation
Line 423: Pleas be specific for which use these reanalysis products are recommended. The reader leaves the study with the question - for what can these SM reanalysis be recommended?
Figure 10: Consider using more positive scoring such as +++ (optimal), ++ (ok) and + (least). No one wants to see "not recommended". Also, ERA5-Land in Humid seems to be rather not recommended according to your results (high bias) but must be considered optimal given there is no better product.
Line 487: See above and add "generally" good agreement.
Line 498: There is no "balanced climate". What climate are you referring to here? I guess: temperate. Please correct.
Citation: https://doi.org/10.5194/hess-2023-224-RC2 -
AC2: 'Reply on RC2', Yanchen Zheng, 20 Dec 2023
Reply to Reviewer #2 Comments
We have categorized the reviewers comments and provided responses. Author responses are in bold and italic.The paper analyses the performance of global reanalysis soil moisture products against soil moisture obtained from cosmic ray neutron sensors. The results are analysed using different performance criteria, region specific and climate specific. The paper addresses highly relevent scientific questions, is exceptionally well structured, provides novel insights thereby presenting novel concepts and setting the global data sets into new context. The conclusions are reached, to the point and substantial.
We are grateful for the insightful and constructive feedback provided by Reviewer 2. We agree that addressing these comments will contribute to enhancing the quality of our work.
However, there are two main improvements to be made before acceptance.
First: I suggest the authors consider to include SMAP Level 4 data as reanalysis product for evaluation. SMAP Level 4 comes as data assimilation product and is considered state-of-the-art soil moisture for the recent years. Without SMAP Level 4, the study seems incomplete.
Line 50: This is well formulated. However, SMAP Level 4 is missing in this list https://doi.org/10.1029/2019MS001729
Line 132: Where is SMAP Level 4? Please give strong reason for not using the most recent Soil Moisture Product https://doi.org/10.1029/2019MS001729 SMAP Level 4 is a reanalysis product as it includes meteorological variables, and satellite observations that are used to update soil moisture in a data assimilation framework.
In fact, I strongly suggest to add SMAP Level 4. As such, the study is out-dated. Adding SMAP Level 4 should be a short exercise - multiplying the scientific impact of this study by a factor. Hopefully SMAP Level 4 performs best amongst all Reanalysis products. Although it is not 20 years length, it is the SM reanalysis product that will be used for recent years rather than any of the other products analyzed. The technical definition of 20+ years (line 42) by a reference of 2005 is not sufficient to not use the most up-to-date SM reanalysis product in this highly relevant global study.Thanks for the above suggestion of including SMAP Level 4 products for evaluation. We agree that SMAP Level 4 product is state-of-the-art soil moisture for recent years. We will have an investigation of this product to check its availability and applicability allied with our objectives.
Second: the terminology in Figure 10 as "highly recommended", "recommended" and "not recommended" must be adapted and if needed must be adapted throughout the text. All reanalysis products show high bias, and hence, the ERA5-Land cannot be highly recommended for humid sites as the authors have done in Figure 10. It should rather be recommended with care. I provide a suggestion below.
Figure 10: Consider using more positive scoring such as +++ (optimal), ++ (ok) and + (least). No one wants to see "not recommended". Also, ERA5-Land in Humid seems to be rather not recommended according to your results (high bias) but must be considered optimal given there is no better product.As for the terminology in Figure 10, we agree that the current words are not appropriate. In our revision, we will change our wording and adapt throughout the paper.
Minor comments:
Line 11: It is recommended to shorten the abstract. Lines 11-19 can be reduced to 3 lines. This will focus the reader on the methods, novelty, and key findings. These are mentioned in the abstract.
Line 21: Although UK has its own network, it reads awkwards to read "sites from UK, Europe, USA and Australia" as a list because the UK is included in Europe and should not be listed beside Europe as separate geogrphic entity. Please not that this list refers to CRNS networks COSMOS-UK, COSMOS-Europe...
Line 318: "low temporal correlation" directly contradicts the abstract line 22: "all reanalysis products exhibit good temporal correlation with the measurements". In the abstract, I suggest to add "products generally exhibit" to weaken this statement in the abstract.
Line 487: See above and add "generally" good agreement.Thanks for pointing these out. We will revise and shorten the abstract carefully in our revision to make it more focus on methods, novelty and key findings. Also we will pay attention to appropriate wording (i.e., adding “products generally exhibit”) and rephrase “sites from UK, Europe, USA and Australia” to “CRNS sites from the COSMOS-UK, COSMOS-Europe, COSMOS USA and CosmOz Australia networks.”
Line 42: Please also list this newer review on reanalysis, were a definition of reanalysis products is given https://doi.org/10.1029/2020RG000715
Thanks for suggesting this literature for the definition of reanalysis products. We will cite this newer paper as reference.
Line 164: This depends on the source of uncertainty. Daily averaging causes loss of signal and proper filtering maintains signal while reducing uncertainty/noise. High measurement uncertainty can be compensated for by applying temporal filtering methods or simple daily averaging e.g. https://doi.org/10.3390/s22239143 .
Thanks for your recommendation of this reference. We will clarify this in our revision.
Figure 2: Add region names and a/b/c/d to the figure ( or names of the networks ).
Thanks for spotting the missing labels of Figure 2, we will make sure to add the region names and a-d label in Figure 2 in our revision.
Line 330: I contradict, a model cannot perform best in all statistical metrics except for bias. High bias produces high MSE. With a poor bias, the MSE should be poor as well. Please clarify.
We will carefully check our statistical metrics values of this and will correct the wording of this sentence.
Line 345: better compared to what? Please specify.
Line 423: Pleas be specific for which use these reanalysis products are recommended. The reader leaves the study with the question - for what can these SM reanalysis be recommended?Thanks for pointing out these unclear sentences. We will carefully re-write these sentences and provide more specific information especially on the reanalysis products recommendations.
Line 383: Please add that vegetation correction were proposed e.g. https://doi.org/10.1002/2014WR016443
Thanks for suggesting this reference. We will add in our reference list.
Line 385: Please discuss in a sentence your study results with these results: https://doi.org/10.5194/hess-21-6201-2017
Thanks for highlighting this reference. We found similarities between our studies and this paper from Beck et al. (2017). In their studies (Table 2) evaluating 22 precipitation datasets, they also found that all the precipitation products tend to capture the monthly variation well but have lower performance in shorter timescales (i.e., Pearson correlation coefficient calculated for 3-day means, R3day). This aligns with our findings that the reanalysis products tend to reproduce the season pattern of the variables well but hard to capture the anomalies. We will add these in our discussion.
All other minor comments:
Line 115: rephrase to "harmonized processing of CRNS datasets".
Line 174: Rephrase towards more active voice: "Spatial scale matching"
Line 184: see comment before.
Line 290: Clarify meaning of "Figure 4cd". And typically it is called "US sites" rather than "USA sites".
Line 306: Clarify the use of "main paper" which implies, there is a secondary paper.
Line 325: Please rephrase. Sites cannot be a reason for low performance. Reasons for low performance can only be process related or of technical nature. A site itself cannot be the reason for low performance.
Line 407: lower performance is preferred to "worse" performance which comes with a very negative connotation
Line 329: CFSv2 or CFSRv2?
Line 498: There is no "balanced climate". What climate are you referring to here? I guess: temperate. Please correct.Thanks for providing the above valuable comments about the wording, we will rephrase and clarify these carefully in our revision. We will provide the point-to-point response and detailed revisions in our later response to reviewers files.
Citation: https://doi.org/10.5194/hess-2023-224-AC2
-
AC2: 'Reply on RC2', Yanchen Zheng, 20 Dec 2023
Status: closed
-
RC1: 'Comment on hess-2023-224', Anonymous Referee #1, 15 Nov 2023
The authors present a noteworthy study that compares CRNS stations across multiple continents with remote sensing/reanalysis soil moisture products. Utilizing the CRSPY tool, the authors reprocess the raw CRNS data, addressing gaps in information, such as the atmospheric moisture correction (absent in the COSMOS USA network), and harmonize the dataset. Undoubtedly, this tool holds promise for advancing the CRNS method and enhancing the global CRNS community. The article is well-crafted, with comprehensive data documentation in both the appendix and online. However, I have some suggestions and comments that should be addressed before publication.
Major Suggestion:
I find some ambiguity regarding the reprocessing of CRNS data with CRSPY, particularly in the computation of the N0 parameter. The main article and supplemental material lack any mention of N0. To calculate N0, each site typically requires at least one gravimetric calibration survey, composed of 12-18 soil profiles within the footprint, sampled every 5 cm down to 30 cm. The gravimetric survey data is then weighted (following Kohli 2015/2017, etc.), and the Desilets 2010 function is inverted to determine N0. Most CRNS sites have gravimetric calibration data for one or more sample periods. Please include a detailed description of the process for estimating N0 at each site, specifying the gravimetric calibration dataset used (single or average of multiple calibrations, etc.). The accuracy of the N0 estimate is crucial, as incomplete or non-representative data may introduce significant bias in the comparison with reanalysis products. Best practices for N0 involve 2-3 calibration dates (Iwema 2015), though this is labor-intensive and not consistently implemented.
I am a bit confused by the description of soil organic carbon. Does this include both the organic carbon and mineral lattice water values? The variation in lattice water was found to be important for many of the original CRNS networks (Zreda 2012 COSMOS, Hawdon 2014 COSMOZ, etc.). The paper and appendix does not include the description of lattice water and should be clarified or added to the metadata.
Comment: The influence of rapidly growing crops on the CRNS observations remains a challenge. There have been several attempts to provide correction factors (either on N0 itself or on the moderated counts) but nothing definitive has been adopted by the community. The authors mention this in the limitations. I hope the community comes to a consensus soon about how best to deal with CRNS data in croplands. The influence of forest biomass seems to be an even more challenging problem but equally important.
Minor comment:
Figure 2. Label the 4 subplots a-d and list what geographical region they are from. This was not clear from the description.
Citation: https://doi.org/10.5194/hess-2023-224-RC1 -
AC1: 'Reply on RC1', Yanchen Zheng, 20 Dec 2023
Reply to Reviewer #1 Comments
Author responses are in bold and italic.The authors present a noteworthy study that compares CRNS stations across multiple continents with remote sensing/reanalysis soil moisture products. Utilizing the CRSPY tool, the authors reprocess the raw CRNS data, addressing gaps in information, such as the atmospheric moisture correction (absent in the COSMOS USA network), and harmonize the dataset. Undoubtedly, this tool holds promise for advancing the CRNS method and enhancing the global CRNS community. The article is well-crafted, with comprehensive data documentation in both the appendix and online. However, I have some suggestions and comments that should be addressed before publication.
Thanks for your valuable and positive comments of our paper. We appreciate your insightful comments, and we are committed to addressing the mentioned points in our revision.
Major Suggestion:
I find some ambiguity regarding the reprocessing of CRNS data with CRSPY, particularly in the computation of the N0 parameter. The main article and supplemental material lack any mention of N0. To calculate N0, each site typically requires at least one gravimetric calibration survey, composed of 12-18 soil profiles within the footprint, sampled every 5 cm down to 30 cm. The gravimetric survey data is then weighted (following Kohli 2015/2017, etc.), and the Desilets 2010 function is inverted to determine N0. Most CRNS sites have gravimetric calibration data for one or more sample periods. Please include a detailed description of the process for estimating N0 at each site, specifying the gravimetric calibration dataset used (single or average of multiple calibrations, etc.). The accuracy of the N0 estimate is crucial, as incomplete or non-representative data may introduce significant bias in the comparison with reanalysis products. Best practices for N0 involve 2-3 calibration dates (Iwema 2015), though this is labor-intensive and not consistently implemented.
I am a bit confused by the description of soil organic carbon. Does this include both the organic carbon and mineral lattice water values? The variation in lattice water was found to be important for many of the original CRNS networks (Zreda 2012 COSMOS, Hawdon 2014 COSMOZ, etc.). The paper and appendix does not include the description of lattice water and should be clarified or added to the metadata.We agree that in the current paper we didn’t mention about the information of N0 and lattice water for CRNS sites. We will add descriptions of these in the paper and also include the N0 and lattice water data for the available sites in the supplementary file.
Comment: The influence of rapidly growing crops on the CRNS observations remains a challenge. There have been several attempts to provide correction factors (either on N0 itself or on the moderated counts) but nothing definitive has been adopted by the community. The authors mention this in the limitations. I hope the community comes to a consensus soon about how best to deal with CRNS data in croplands. The influence of forest biomass seems to be an even more challenging problem but equally important.
As for the influence of vegetation effects on CRNS observations, thanks for agreeing to this point. We share the hope that the research community will collectively work towards developing more effective solutions to address and mitigate these effects in future studies.
Minor comment:
Figure 2. Label the 4 subplots a-d and list what geographical region they are from. This was not clear from the description.Thanks for spotting the missing labels of Figure 2, we will make sure to add the geographical region and a-d label in Figure 2 in our revision.
Citation: https://doi.org/10.5194/hess-2023-224-AC1
-
AC1: 'Reply on RC1', Yanchen Zheng, 20 Dec 2023
-
RC2: 'Comment on hess-2023-224', Anonymous Referee #2, 20 Nov 2023
The paper analyses the performance of global reanalysis soil moisture products against soil moisture obtained from cosmic ray neutron sensors. The results are analysed using different performance criteria, region specific and climate specific. The paper addresses highly relevent scientific questions, is exceptionally well structured, provides novel insights thereby presenting novel concepts and setting the global data sets into new context. The conclusions are reached, to the point and substantial.
However, there are two main improvements to be made before acceptance.
First: I suggest the authors consider to include SMAP Level 4 data as reanalysis product for evaluation. SMAP Level 4 comes as data assimilation product and is considered state-of-the-art soil moisture for the recent years. Without SMAP Level 4, the study seems incomplete.
Second: the terminology in Figure 10 as "highly recommended", "recommended" and "not recommended" must be adapted and if needed must be adapted throughout the text. All reanalysis products show high bias, and hence, the ERA5-Land cannot be highly recommended for humid sites as the authors have done in Figure 10. It should rather be recommended with care. I provide a suggestion below.
Minor comments:
Line 11: It is recommended to shorten the abstract. Lines 11-19 can be reduced to 3 lines. This will focus the reader on the methods, novelty, and key findings. These are mentioned in the abstract.Line 21: Although UK has its own network, it reads awkwards to read "sites from UK, Europe, USA and Australia" as a list because the UK is included in Europe and should not be listed beside Europe as separate geogrphic entity. Please not that this list refers to CRNS networks COSMOS-UK, COSMOS-Europe...
Line 42: Please also list this newer review on reanalysis, were a definition of reanalysis products is given https://doi.org/10.1029/2020RG000715
Line 50: This is well formulated. However, SMAP Level 4 is missing in this list https://doi.org/10.1029/2019MS001729
Line 115: rephrase to "harmonized processing of CRNS datasets".
Line 132: Where is SMAP Level 4? Please give strong reason for not using the most recent Soil Moisture Product https://doi.org/10.1029/2019MS001729 SMAP Level 4 is a reanalysis product as it includes meteorological variables, and satellite observations that are used to update soil moisture in a data assimilation framework.
In fact, I strongly suggest to add SMAP Level 4. As such, the study is out-dated. Adding SMAP Level 4 should be a short exercise - multiplying the scientific impact of this study by a factor. Hopefully SMAP Level 4 performs best amongst all Reanalysis products. Although it is not 20 years length, it is the SM reanalysis product that will be used for recent years rather than any of the other products analyzed. The technical definition of 20+ years (line 42) by a reference of 2005 is not sufficient to not use the most up-to-date SM reanalysis product in this highly relevant global study.
Line 164: This depends on the source of uncertainty. Daily averaging causes loss of signal and proper filtering maintains signal while reducing uncertainty/noise. High measurement uncertainty can be compensated for by applying temporal filtering methods or simple daily averaging e.g. https://doi.org/10.3390/s22239143 .
Line 174: Rephrase towards more active voice: "Spatial scale matching"
Line 184: see comment before.
Figure 2: Add region names and a/b/c/d to the figure ( or names of the networks ).
Line 290: Clarify meaning of "Figure 4cd". And typically it is called "US sites" rather than "USA sites".
Line 306: Clarify the use of "main paper" which implies, there is a secondary paper.
Line 318: "low temporal correlation" directly contradicts the abstract line 22: "all reanalysis products exhibit good temporal correlation with
the measurements". In the abstract, I suggest to add "products generally exhibit" to weaken this statement in the abstract.Line 325: Please rephrase. Sites cannot be a reason for low performance. Reasons for low performance can only be process related or of technical nature. A site itself cannot be the reason for low performance.
Line 329: CFSv2 or CFSRv2?
Line 330: I contradict, a model cannot perform best in all statistical metrics except for bias. High bias produces high MSE. With a poor bias, the MSE should be poor as well. Please clarify.Line 345: better compared to what? Please specify.
Line 383: Please add that vegetation correction were proposed e.g. https://doi.org/10.1002/2014WR016443
Line 385: Please discuss in a sentence your study results with these results: https://doi.org/10.5194/hess-21-6201-2017
Line 407: lower performance is preferred to "worse" performance which comes with a very negative connotation
Line 423: Pleas be specific for which use these reanalysis products are recommended. The reader leaves the study with the question - for what can these SM reanalysis be recommended?
Figure 10: Consider using more positive scoring such as +++ (optimal), ++ (ok) and + (least). No one wants to see "not recommended". Also, ERA5-Land in Humid seems to be rather not recommended according to your results (high bias) but must be considered optimal given there is no better product.
Line 487: See above and add "generally" good agreement.
Line 498: There is no "balanced climate". What climate are you referring to here? I guess: temperate. Please correct.
Citation: https://doi.org/10.5194/hess-2023-224-RC2 -
AC2: 'Reply on RC2', Yanchen Zheng, 20 Dec 2023
Reply to Reviewer #2 Comments
We have categorized the reviewers comments and provided responses. Author responses are in bold and italic.The paper analyses the performance of global reanalysis soil moisture products against soil moisture obtained from cosmic ray neutron sensors. The results are analysed using different performance criteria, region specific and climate specific. The paper addresses highly relevent scientific questions, is exceptionally well structured, provides novel insights thereby presenting novel concepts and setting the global data sets into new context. The conclusions are reached, to the point and substantial.
We are grateful for the insightful and constructive feedback provided by Reviewer 2. We agree that addressing these comments will contribute to enhancing the quality of our work.
However, there are two main improvements to be made before acceptance.
First: I suggest the authors consider to include SMAP Level 4 data as reanalysis product for evaluation. SMAP Level 4 comes as data assimilation product and is considered state-of-the-art soil moisture for the recent years. Without SMAP Level 4, the study seems incomplete.
Line 50: This is well formulated. However, SMAP Level 4 is missing in this list https://doi.org/10.1029/2019MS001729
Line 132: Where is SMAP Level 4? Please give strong reason for not using the most recent Soil Moisture Product https://doi.org/10.1029/2019MS001729 SMAP Level 4 is a reanalysis product as it includes meteorological variables, and satellite observations that are used to update soil moisture in a data assimilation framework.
In fact, I strongly suggest to add SMAP Level 4. As such, the study is out-dated. Adding SMAP Level 4 should be a short exercise - multiplying the scientific impact of this study by a factor. Hopefully SMAP Level 4 performs best amongst all Reanalysis products. Although it is not 20 years length, it is the SM reanalysis product that will be used for recent years rather than any of the other products analyzed. The technical definition of 20+ years (line 42) by a reference of 2005 is not sufficient to not use the most up-to-date SM reanalysis product in this highly relevant global study.Thanks for the above suggestion of including SMAP Level 4 products for evaluation. We agree that SMAP Level 4 product is state-of-the-art soil moisture for recent years. We will have an investigation of this product to check its availability and applicability allied with our objectives.
Second: the terminology in Figure 10 as "highly recommended", "recommended" and "not recommended" must be adapted and if needed must be adapted throughout the text. All reanalysis products show high bias, and hence, the ERA5-Land cannot be highly recommended for humid sites as the authors have done in Figure 10. It should rather be recommended with care. I provide a suggestion below.
Figure 10: Consider using more positive scoring such as +++ (optimal), ++ (ok) and + (least). No one wants to see "not recommended". Also, ERA5-Land in Humid seems to be rather not recommended according to your results (high bias) but must be considered optimal given there is no better product.As for the terminology in Figure 10, we agree that the current words are not appropriate. In our revision, we will change our wording and adapt throughout the paper.
Minor comments:
Line 11: It is recommended to shorten the abstract. Lines 11-19 can be reduced to 3 lines. This will focus the reader on the methods, novelty, and key findings. These are mentioned in the abstract.
Line 21: Although UK has its own network, it reads awkwards to read "sites from UK, Europe, USA and Australia" as a list because the UK is included in Europe and should not be listed beside Europe as separate geogrphic entity. Please not that this list refers to CRNS networks COSMOS-UK, COSMOS-Europe...
Line 318: "low temporal correlation" directly contradicts the abstract line 22: "all reanalysis products exhibit good temporal correlation with the measurements". In the abstract, I suggest to add "products generally exhibit" to weaken this statement in the abstract.
Line 487: See above and add "generally" good agreement.Thanks for pointing these out. We will revise and shorten the abstract carefully in our revision to make it more focus on methods, novelty and key findings. Also we will pay attention to appropriate wording (i.e., adding “products generally exhibit”) and rephrase “sites from UK, Europe, USA and Australia” to “CRNS sites from the COSMOS-UK, COSMOS-Europe, COSMOS USA and CosmOz Australia networks.”
Line 42: Please also list this newer review on reanalysis, were a definition of reanalysis products is given https://doi.org/10.1029/2020RG000715
Thanks for suggesting this literature for the definition of reanalysis products. We will cite this newer paper as reference.
Line 164: This depends on the source of uncertainty. Daily averaging causes loss of signal and proper filtering maintains signal while reducing uncertainty/noise. High measurement uncertainty can be compensated for by applying temporal filtering methods or simple daily averaging e.g. https://doi.org/10.3390/s22239143 .
Thanks for your recommendation of this reference. We will clarify this in our revision.
Figure 2: Add region names and a/b/c/d to the figure ( or names of the networks ).
Thanks for spotting the missing labels of Figure 2, we will make sure to add the region names and a-d label in Figure 2 in our revision.
Line 330: I contradict, a model cannot perform best in all statistical metrics except for bias. High bias produces high MSE. With a poor bias, the MSE should be poor as well. Please clarify.
We will carefully check our statistical metrics values of this and will correct the wording of this sentence.
Line 345: better compared to what? Please specify.
Line 423: Pleas be specific for which use these reanalysis products are recommended. The reader leaves the study with the question - for what can these SM reanalysis be recommended?Thanks for pointing out these unclear sentences. We will carefully re-write these sentences and provide more specific information especially on the reanalysis products recommendations.
Line 383: Please add that vegetation correction were proposed e.g. https://doi.org/10.1002/2014WR016443
Thanks for suggesting this reference. We will add in our reference list.
Line 385: Please discuss in a sentence your study results with these results: https://doi.org/10.5194/hess-21-6201-2017
Thanks for highlighting this reference. We found similarities between our studies and this paper from Beck et al. (2017). In their studies (Table 2) evaluating 22 precipitation datasets, they also found that all the precipitation products tend to capture the monthly variation well but have lower performance in shorter timescales (i.e., Pearson correlation coefficient calculated for 3-day means, R3day). This aligns with our findings that the reanalysis products tend to reproduce the season pattern of the variables well but hard to capture the anomalies. We will add these in our discussion.
All other minor comments:
Line 115: rephrase to "harmonized processing of CRNS datasets".
Line 174: Rephrase towards more active voice: "Spatial scale matching"
Line 184: see comment before.
Line 290: Clarify meaning of "Figure 4cd". And typically it is called "US sites" rather than "USA sites".
Line 306: Clarify the use of "main paper" which implies, there is a secondary paper.
Line 325: Please rephrase. Sites cannot be a reason for low performance. Reasons for low performance can only be process related or of technical nature. A site itself cannot be the reason for low performance.
Line 407: lower performance is preferred to "worse" performance which comes with a very negative connotation
Line 329: CFSv2 or CFSRv2?
Line 498: There is no "balanced climate". What climate are you referring to here? I guess: temperate. Please correct.Thanks for providing the above valuable comments about the wording, we will rephrase and clarify these carefully in our revision. We will provide the point-to-point response and detailed revisions in our later response to reviewers files.
Citation: https://doi.org/10.5194/hess-2023-224-AC2
-
AC2: 'Reply on RC2', Yanchen Zheng, 20 Dec 2023
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 409 | 128 | 24 | 561 | 78 | 16 | 16 |
- HTML: 409
- PDF: 128
- XML: 24
- Total: 561
- Supplement: 78
- BibTeX: 16
- EndNote: 16
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1