Heterogeneity measures in hydrological frequency analysis: review and new developments

Requena, Ana I.; Chebana, Fateh; Ouarda, Taha B. M. J.

doi:https://doi.org/10.5194/hess-21-1651-2017

Articles | Volume 21, issue 3

https://doi.org/10.5194/hess-21-1651-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/hess-21-1651-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 21, issue 3

Research article

|

21 Mar 2017

Research article |

| 21 Mar 2017

Heterogeneity measures in hydrological frequency analysis: review and new developments

Ana I. Requena, Fateh Chebana, and Taha B. M. J. Ouarda

Download

Final revised paper (published on 21 Mar 2017)
Preprint (discussion started on 05 Apr 2016)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Heterogeneity measures in hydrological studies: review and new developments', Claudia Rojas-Serna, 07 Jun 2016
- AC1: 'Authors' reply to referee comment RC1', Ana Requena, 21 Jun 2016
RC2: 'Interactive comment on “Heterogeneity measures in hydrological studies: review and new developments” by A. I. Requena et al.', Anonymous Referee #2, 13 Jul 2016
- AC3: 'Authors' reply to referee comment RC2', Ana Requena, 29 Aug 2016
RC3: 'Heterogeneity measures in hydrological studies: review and new developments', Anonymous Referee #3, 16 Jul 2016
- AC2: 'Authors' reply to referee comment RC3', Ana Requena, 29 Aug 2016

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (08 Sep 2016) by Stacey Archfield

AR by Ana Requena on behalf of the Authors (31 Oct 2016) Author's response Manuscript

ED: Referee Nomination & Report Request started (17 Nov 2016) by Stacey Archfield

RR by Anonymous Referee #2 (04 Dec 2016)

RR by William Farmer (03 Jan 2017)

Suggestions for revision or reasons for rejection

The authors have provided a scientifically-sound and well-written manuscript presenting, through controlled, scientific experiments using synthetically generated data, a novel technique for assessing the heterogeneity of a region. The authors have sufficiently responded to all previous reviewer comments and the body of the manuscript has improved. Most of my concerns are minor, but a will address my most important concerns first.

The editor and the reviewers called for a section using “real data” or observational data. I think this has been added, but I actually think, in its present form, it detracts from the manuscript. The synthetic experiments provide a more controlled justification of the novel method. When observational data is presented we are confronted with the problem that the true underlying condition is completely unknown. Therefore, exercises based on observational data must be cleverly defined so as to best approximate an unknown truth; this is an incredibly challenging task. For example, no one can definitively state that one of the three observational approaches to regionalization presented here is inherently better than any other. One can make a strong argument, and the authors have attempted to ensure that some of the approaches seem unlikely, but the truth is not known. Therefore, the result cannot be taken as inconclusive evidence. In this way, the introduction of observational data detracts from the neat, controlled experiments using synthetic data.

I am not advocating for the rejection of observational data. Instead, I propose using this data or another constructed example to discuss the limitations of this approach in practice. The authors suggest that the optimal metric would identify approach A as optimal. What factors might affect this discovery in practice? Beyond the excellent sensitivity analysis, are there external uncertainties that may affect your, by analogy, Type I and Type II errors?

In this same vein, I’d like to see the authors expand Table 4. While the GI metric identifies A when a simple average is used, the more-appropriate weighted average identifies approach B as optimal. Furthermore, the weighted-averaging of GI suggests that A, B nor C are less heterogeneous than using the entire region. Only the RRMSE of the 100-year event identifies approach A consistently. What causes this difference? Is it meaningful? (NOTE: When I computed weighted averages for GI I got 0.093, 0.092, and 0.093 for A, B, and C respectively. For RRMSE of the 100-year event, I got 16.20, 18.00, and 17.20; for the 10-year event I got 5.89, 5.83, and 5.61.)

Let me close my main comments by saying that the remainder of the manuscript is very well written and the scientific procedure is well devised and executed. The findings are both novel and interesting, meriting publication. In coordination with the editor’s opinion, I would advise revising the narrative of the observational example.

The following a several more minor comments that I feel may be worthy of consideration towards improving the manuscript. They are presented in loose sequential order.

P1, L27: Is hydrologic transfer from similar sites the only approach?

P3, L7: What were the results of Wright et al. (2015)? How are they useful to this discussion?

P4, L29: This statement is unclear to me. While L-CV and L-skew may be correlated, the heterogeneity rate appears to be more akin to variability, being a range divided by an average. Is there evidence to support similar variabilities? Can you clarify this statement? At first I thought you meant that L-CV equaled L-skew, but this is certainly not correct. I mention my misunderstanding only as an example the need for clarity.

P5, L1: In this paragraph, it might be clearer to use display equations rather than in-line equations.

P6, L1: Would there be value to considering more heterogeneity rates?

P11, E11: Could the average L-CV be replaced by the regional L-CV? What would be the effect?

P13, L2: It may be useful to consider revising the presentation of results to appear more objective. For example, “favorable” and “bad” behavior seems to approach a subjective presentation. I often struggle with this in my own writing, so I only pass this comment along in case you find it useful.

P13, L24: This error rate seems to be an analog to Type I and Type II errors in hypothesis testing. Would that be a useful way to think about this approach? Does one metric provide a smaller error rate in either case?

Thank you for the opportunity to review this manuscript. It is very well executed and I look forward to its publication. Please, do not hesitate to contact me if you would like to discuss my comments further.

Thank you,
William Farmer
Research Scientist
U.S. Geological Survey

Hide

RR by Keith Sawicz (06 Jan 2017)

Suggestions for revision or reasons for rejection

The manuscript titled "Heterogeneity measures in hydrological frequency analysis: review and new developments." The study explores measures of heterogeneity on synthesized data, and I believe that it’s contribution is value to the hydrologic sciences. The authors have improved the clarity of data synthesis methods from its last iteration, however I believe that additional revision is necessary before publication. I have included specific comments as examples of how to revise the manuscript for clarity and simplicity.

Page 1 Line 21: “…used criteria in assessing…” should be changed to “used criteria to assess”
Page 1 Line 28: “homogenous regions” has not been defined before this point. Care needs to be taken to use this nomenclature and I think that it is being used inappropriately in this case. Consider either replacing homogeneous with terminology used in the previous sentence (“hydrologically similar gauged sites”). Page 2 Line 3 has a definition of regional homogeneity that defines what the term means. Please define this term before using terminology like “homogeneous regions.”
Page 2 Line 9: Change “Known statistical tools” to “Traditional statistical tools.”
Page 2 Line 21-22: I do not believe that there is a definition of what you mean by “the quantile estimate.” Please define prior to this statement.
Page 3 Line 11: The sentence containing “As a consequence of the non-availability…from the quantile estimate step.” is overly complex. Consider simplifying the sentence structure to convey a clear point.
Page 3 Line 26: I am not sure which approach you are referring to here when you say “…the latter approach…” The structure just before this sentence does not make it clear.
Page 5 Line 9: The sentence containing “the value of tau3 is (usually) omitted, as tau3 is considered to have the same value as tau.” seems to either be a finding from this study or an observation from other studies. If it is a finding of this study, this information is appropriate in the results, not the methods. If it is an observation from previous studies, a citation is needed.
Page 5 Line 14: I agree that with the sentence containing “…the use of simulated data in the assessment of new techniques in regional frequency analysis is a very well established approach…“ in that simulated data can be used to generally test new techniques because you can generate data with simple assumptions and not need to address uncertainty or bias in your data. However, while it is clear that data was simulated to test the new approach, it is not clear to me what kind of data was simulated. Was streamflow time series generated? Were synthetic flood frequency curves generated and sampled? You do not need to necessarily include the data that you generated, but it is important to specify the steps that you used to generate said data. The assumption is that when these steps are followed, you will arrive at the same conclusions of the paper even if the exact same data was not synthesized.
Page 5 Line 15: "Indeed, this is the only way to deal with issues related to data quality". I don't agree with this statement. In order to simulate data, you are making assumptions about hydrologic systems by the GEV distribution. Like is mentioned in the manuscript, data simulation is commonly used to test new approaches, but this is usually to address sampling bias and remove features of real data such as data availability/length and measurement uncertainty.
Page 8 Line 9: “Heterogeneity measures”. The previous section is titled ‘assessment of heterogeneity measures,’ but this section mentions the measures. Logically, I think you should introduce what you are going to compare before you talk about how you will compare them, unless there is a good reason otherwise.
Page 12 Line 11: In the sentence that says “all considered measures show a good behavior regarding the heterogeneity rate y, as their values increase with y.”, good behavior is not a good way of describing this. These measures seem to be positively correlated with an increasing y, which would mean that all of the measures can indicate heterogeneity. However, the sentence as it is written is imprecise and vague. This is an example of sentence structure that should be simplified to better convey what you precisely mean and not rely on the reader to interpret what you mean.

Table 1: I have not found where the definition of what is considered good, acceptable, or bad. Was there a numerical threshold determined for each of these? Are they purely qualitative? This table is comparing different measures of heterogeneity, so showing the quantitative thresholds that you used is useful to the reader.
Figures (General): Generally, I find the figures to be labeled well and clear to understand. There is a legend in the figures that identify the difference between gray and black lines. One addition to this legend should be a mention of gray and black colors with respect to y values.

Hide

ED: Publish subject to minor revisions (further review by Editor) (21 Jan 2017) by Stacey Archfield

AR by Ana Requena on behalf of the Authors (07 Feb 2017) Author's response Manuscript

ED: Publish as is (22 Feb 2017) by Stacey Archfield

AR by Ana Requena on behalf of the Authors (02 Mar 2017)

Short summary

The notion of a measure to quantify the degree of heterogeneity of a region from which information is required to estimate the magnitude of events at ungauged sites is introduced. These heterogeneity measures are needed to compare regions, evaluate the impact of particular sites, and rank the performance of delineating methods. A framework to define and assess their desirable properties is proposed. Several heterogeneity measures are presented and/or developed to be assessed, giving guidelines.