the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Seasonal forecasting of lake water quality and algal bloom risk using a continuous Gaussian Bayesian network
- Norwegian Institute for Water Research (NIVA), 0349 Oslo, Norway
- Norwegian Institute for Water Research (NIVA), 0349 Oslo, Norway
Abstract. Freshwater management is challenging, and advance warning that poor water quality was likely, a season ahead, could allow for preventative measures to be put in place. To this end, we developed a Bayesian network (BN) for seasonal lake water quality prediction. BNs have become popular in recent years, but the vast majority are discrete. Here we developed a Gaussian Bayesian network (GBN), a simple class of continuous BN. The aim was to forecast, in spring, total phosphorus (TP), chlorophyll-a (chl-a), cyanobacteria biovolume and water colour for the coming growing season (May–October) in lake Vansjø in southeast Norway. To develop the model, we first identified controls on inter-annual variability in water quality using correlations, scatterplots, regression tree based feature importance analysis and process knowledge. Key predictors identified were lake conditions the previous summer, a TP control on algal variables, a colour-cyanobacteria relationship, and weaker relationships between precipitation and colour and between wind and chl-a. These variables were then included in the GBN and conditional probability densities were fitted using observations (≤ 39 years). GBN predictions had R2 values of 0.37 (cyanobacteria) to 0.75 (colour) and classification errors of 32 % (TP) to 13 % (cyanobacteria). For all but lake colour, including weather nodes did not improve predictive performance (assessed through cross validation). Overall, we found the GBN approach to be well-suited to seasonal water quality forecasting. It was straightforward to produce probabilistic predictions, including the probability of exceeding management-relevant thresholds. The GBN could be purely parameterised using observed data, despite the small dataset. This wasn’t possible using a discrete BN, highlighting a particular advantage of using GBNs when sample sizes are small. Although low interannual variability and high temporal autocorrelation in the study lake meant the GBN performed similarly to a seasonal naïve forecast, we believe the forecasting approach presented could be useful in areas with higher sensitivity to catchment nutrient delivery and seasonal climate, and for forecasting at shorter time scales (e.g. daily to monthly). Despite the parametric constraints of GBNs, their simplicity, together with the relative accessibility of BN software with GBN handling, means they are a good first choice for BN development, particularly when datasets for model training are small.
Leah Jackson-Blake et al.
Status: closed
-
RC1: 'Comment on hess-2021-621', Anonymous Referee #1, 21 Jan 2022
Overview
This paper presents a Gaussian Bayesian Network (GBN) for seasonal lake water quality (TP, chl-a, cyanobacteria and colour) forecasting. The GBN was developed and applied to Lake Vansjø in southeast Norway. The GBN was found well-suited for seasonal water quality forecasting and could be parameterised purely using observed data, despite this dataset being small. The forecasting performance of the GBN was assessed using a cross validation scheme, and the performance was also compared to that of a discrete BN (with the same structure) and a naïve forecast model; it was found that the 3 models performed similarly largely due to low interannual variability and high temporal autocorrelation in the study lake.
General comment
Overall, I think this is an interesting study that is very relevant to the HESS special issue. Although the forecasting results with the GBN were considered a mixed success at the study site, I do agree with the authors that the GBN seems to be a sensible and promising approach for water quality forecasting, and I think that by sharing all the code on GitHub the authors have provided a very useful tool for others to use and adapt. I very much enjoyed reading the paper which is both well-written and well-presented, and I believe it can be accepted after some minor revisions.
Below are my comments which I hope the authors will find useful. Lykke til!
Specific comments
- My main concern is related to the discrete BN and the comparison to the GBN. It is interesting that the discrete BN did a mixed job of representing the relationships, however, I don’t understand why this happens and I think this could be elaborated on further. Specific comments in relation to this:
(i) I’m not sure how the method you used to fit the CPTs works, but considering you have a small dataset and that you are using flat priors, I’m surprised that the fitted CPT in Table 6 seems to suggest that the evidence was strong (i.e., most of parent state combinations results in low-high probabilities of around 99%-1% and 95%-5% or vice versa). Intuitively, I would have thought that the probabilities would still be influenced by the flat prior given the small dataset, but the priors have been completely “outweighed” by the data. To me this suggest that there is something odd about the discretisation of the data and/or the target node states.
(ii) I had a brief look at what I believe is your discretised input data files on Github (.. \BayesianNetwork\Data\DataMatrices\Discretized\), and I think these look a bit strange (although I appreciate these may not be the final version). First of all, the ‘colour_prevSummer’ node seems to have been given 3 states (L, M, H) contrary to what is stated in the manuscript. It also looks like the value for ‘chla’ does not always match the value of ‘chla_prevSummer’ the previous year. The same is the case for ‘colour’ and ‘colour_prevSummer’. I would urge the authors to double check these data files and see if this possibly explain (at least partly) the results of the discrete BBN.
(iii) Finally, I wonder if it would not have been better to use expert opinion to reflect the priors in the discrete network before training, especially as you have a small dataset? To me this would seem sensible, and you already use expert opinion to inform the structure of the network. I also wonder whether you could just have discretised your GBN after it was created (in software like Netica and Genie you can specify continuous distributions and then subsequently discretise these distribution) and how the discretised model would then perform? - I’m not sure I fully understand how the leave-one-out cross validation works and I think it would be great if the authors could make this a bit clearer in section 2.7.1. Do you leave one data point (i.e., a year?) out at the time and then fit the GBN to the remaining data and see how well the GBN predicts the target node time-series? Or how well the GBN predicts the data point that was left out? Or something else? I also don’t really understand why the cross validation is stochastic and why it was run a default 20 times.
Minor comments
- Author name: I believe it should be James E. Sample. Alternatively, change JES to JS in author contributions (L670).
- L21: change “wasn’t” to “was not”
- L63-64: maybe worth explaining what polymictic and dimictic lakes are; at least I’m not familiar with these terms.
- Figure 1: where is the outlet from Vanemfjorden? At Moss River?
- L127+: Can you explain briefly why Vanemfjorden with its short residence time is more susceptible to eutrophication and cyano blooms than Storefjorden, and why it does not seem to be related to the major input source from River Hobol?
- L176: Should it be 1998-2013? At least in L179 you seem to suggest NIVA for 2013 as well.
- L188: specify that it is River Hobol
- L192: Change “As the aim” to “The aim”. Alternatively combine the two sentences in L192-195 and remove “therefore” on L194.
- Figure 2: You could consider plotting error bars to give an idea of the variation in the different parameters.
- L227-229: I’m not sure I understand why these features would have to be included as latent variables. Because they are not measured? From Figure 1, it looks like there are monitoring stations in the eastern lake basin (the same as Storefjorden?), so would you not have water quality data from here?
- Table 1 and Table 2: I find it slightly confusing what features are included. Are all the features for the 6-month growing season as well as for the previous winter season (Nov-Apr), i.e., the number of features used for all variables are at least 2x13? Looking at Table 2, and if I understand the caption correctly, it looks like cyano has 8 additional features, so 34 in total (not 33).
- Table 2: Are the features chl-a_prev, cyano chl-a and cyano_prevSummer for the lake?
- L293-300: I think this would be better presented as a table, where you clearly state what is defined as Low and High in the model. The specific comments related to the water quality parameter in question could then be added in a separate column (e.g., that L and H for TP is in fact lower and upper moderate and so on).
- L304+: I don’t follow this part of the discretisation process and why you get unbalanced class sizes. Are the variables still transformed in the discrete version and fairly normal?
- L348+ and Figure 3: Is the relationship between number of calm days and TP negative? To me it looks like the two are positively correlated.
- L355: Are wind speed (winter_wind) and TP(PS) positively correlated?
- Figure 3-6: What are the bell-shaped curves and how were they derived?
- Figure 7: Is TP_prev supposed to be linked to chl-a_prev? If so, should chl-a_prev not have a beta1_TP_prev coefficient?
- L456: Should it not say: “For parentless nodes…”? Some of your nodes are both parent and child nodes (e.g. lake TP is the parent of lake chl-a but the child of TP_prev)
- L526: As you say, this bias in cyano is likely due to the box-cox transformation. Rather than the mean, would it not have been better to use the median (or mode)? Also, did you calculate the mean before or after back-transformation?
- L656: change wasn’t to was not
-
AC1: 'Reply on RC1', Leah Jackson-Blake, 09 Feb 2022
Many thanks for a very useful review, we really appreciate your careful reading of the paper. You have made some very good comments, as well as pointing out some errors and areas where our descriptions were lacking.
In the attached Reviewer1_response.pdf file we provide comments in response to every point raised and suggestions for changes we would make to the text during revisions. Main points in this response include:
Response to major comments:
Firstly, it is great that you think that the data and code might be a useful tool for people. The active repository is a bit of a live (and therefore somewhat messy) workplace, and I will aim to produce a cut-down working version of the code and data used in the paper and archive it in e.g. Zenodo before final publication.
1.(i) We should have provided more detail in the text on the method used for fitting the discrete network. We used the default uniform prior in BNLearn, the very simple so-called ‘K2’ prior, which just adds 1 to the count of every state before counting how often each state of the variable occurs in the data (conditionally on the parent states, where relevant) to estimate the values of the CPDs. The prior doesn’t therefore have much weight compared to the data. However, we could instead have specified an imaginary sample size (iss) >1 (where the iss, or pseudo-counts, are equivalent to having observed iss uniform samples of each variable). Use of a higher iss would have resulted in a smoother (and potentially more realistic) posterior, and I suggest we explore this when revising.
1. (ii) Thanks for digging into the data! This comment can be split into two:
- You’re right, we used 3 states for colour_prevSummer, not the two that we said in the manuscript (and that were used for all other variables), and the text needs correcting. The hope was that in using three classes we would make the most of the extremely strong correlation between colour_prevSummer and colour.
- Yes, the class of e.g. chl-a in 2018 may not be the same as the class of chl-a_prevSummer in 2019 (and likewise for the other lake water quality variables), despite these summarising the same lake growing season. This is because we were tied to using WFD-relevant thresholds to discretize lake TP, chl-a and cyanobacteria for the current season which we aimed to forecast, to make them management-relevant. However, for all other features, and including lake observations from the previous summer, we could instead use a discretization method that would give us better predictive power. In our case, we used regression trees between parent and child variables to pick the thresholds to use in discretization of the parents. We do say this very briefly in Section 2.6, last paragraph, but I suggest we add extra text to clarify and justify the method, including more text to describe the regression-tree based discretization (and its limitations).
1. (iii) Using expert opinion to inform the priors in the discrete network would indeed likely have given better results. However, it would not then have been a fair test compared with the GBN. Your second point here is a good one, and is something we would incorporate in a revised “Discussion”: rather than using a GBN, a discrete network could have been used where continuous distributions were specified and then discretized. This would have resolved the small sample size issue, and I think it should give near identical results to the GBN (assuming normal distributions were assumed), and would not have the same parametric constraints as other continuous distributions could be specified instead. Although it is a slightly clunky solution compared to just developing a GBN, it could be a good alternative for people who use software that does not have GBN capabilities.
2. The description of the cross validation (CV) scheme used needs updating and we will attempt to improve it to make it clearer what was involved. In fact, the method description is currently slightly outdated, which we apologize for. We used k-fold CV, not leave-one-out CV, but with a high value of k (20) so that it approached leave-one-out CV for cyanobacteria (n=23). In k-fold CV the data are randomly assigned to the k subsets, hence the stochastic element.
Response to minor comments:
20. This comment made us reassess our approach to back-transforming the cyanobacteria predictions (which were Box-Cox transformed for fitting the GBN and producing predictions, and then the expected value was back-transformed to the original data scale). Extra reading has made us realise that straight back transforming from a Box-Cox results in a prediction of the median (e.g. Hyndman and Athanasopoulos, 2018, Chapter 3.2). We will explore using a bias-adjusted back transformation to instead calculate the forecasted mean on the original data scale instead. This should reduce the bias in the GBN predictions and may therefore make the GBN perform better, potentially altering some of our results and their interpretation.
References:
Hyndman, Rob J., and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018. https://otexts.com/fpp2/
-
RC6: 'Reply on AC1', Anonymous Referee #1, 28 Feb 2022
Thanks to the authors for the thorough response to my comments. All make sense to me and I'm happy with the proposed changes.
With respect to comment 20 and bias adjustment of box-cox, this was exactly what I was getting at. The following paper might be relevant, which provides a Taylor series expansion for the mean and variance of a "box-cox normal" distribution:
Peter K. Kitanidis & Kuo-Fen Shen (1996): Geostatistical interpolation of chemical concentration. Advances in Water Resources, Vol. 19, No. 6, pp. 369-318, 1996.
This paper also shows how to include the box-cox parameter in the likelihood function for parameter estimation (essentially punishes higher degree of transformation, i.e. lower values of the box cox transformation parameter); although, I'm not sure this latter part is relevant to your study .
Good luck with the revisions
- AC6: 'Reply on RC6', Leah Jackson-Blake, 28 Feb 2022
- My main concern is related to the discrete BN and the comparison to the GBN. It is interesting that the discrete BN did a mixed job of representing the relationships, however, I don’t understand why this happens and I think this could be elaborated on further. Specific comments in relation to this:
-
RC2: 'Comment on hess-2021-621', Anonymous Referee #2, 16 Feb 2022
General comments:
This study presents an application of a Continuous Bayesian Network (CBN) to seasonal (6-month average) algal forecasting in a northern lake. This is likely the first use of CBN for this purpose. In general, the model performs similarly to a traditional (discretized) BN and a naïve model (using the mean from the previous year). It could be a good fit for this special issue, but I do have several concerns, as outlined below.
I’m not really sure that there is a strong contribution, as the CBN does not perform particularly well. Also, the model appears to be based on existing software (an R package), so there isn’t new methods development. If the objective of the study is to provide a thorough demonstration of CBNs for algal bloom modeling, that could potentially be an important contribution. In this case, I’d like to see more demonstrations of how the CBN approach (e.g., Figure 7) can be advantageous for studying a system or supporting management. In my opinion, the current discussion is too focused on skill assessment (e.g., R2), which probably doesn’t do justice to the CBN approach. Also, probabilistic predictions using various linear covariates can also be obtained through multiple linear regression (frequentist or Bayesian), so why use a CBN? I think there are potentially good reasons for using a CBN, but they aren’t compellingly demonstrated in the current manuscript. Also, I’d like to see more discussion of how this effort compares to other CBN (or BN) applications for water quality or environmental sciences, more broadly.
Major comments:
The paper includes a tangential analysis on making predictions at smaller time scales (e.g., Lines 208-215). I recommend removing this material, as it doesn’t seem relevant to the main focus of this paper (no CBN was used). Furthermore, this additional analysis doesn’t provide new insights (that aren’t available through existing phytoplankton literature). It seems a bit “tacked on”. If you do keep this analysis, the data should be presented (as in Figure 2 for the six-month model).
The variable selection process seems ad hoc (Section 3.1.1), making it somewhat hard to follow and likely difficult to reproduce. Some of the explanations seem questionable. For example, the article cites previous literature showing that “windier summers” are relevant, but the CBN uses winds from the previous 6 months (prior to summer), right? I have two general suggestions. First use clear and consistent terminology that clarifies which time periods you are talking about (also use consistent notation across the different figures and tables). Second, drop wind from the 6-month analysis altogether. Much of the text is a somewhat tortuous explanation (at least for this reader) of reasons to include/exclude wind speed, while in reality, the authors readily acknowledge that wind speed is only relevant at smaller time scales (e.g., Lines 443-445: “wind would likely only have an immediate and relatively short-lived effect…”), not ~6 months in advance.
Detail-oriented comments:
Line 11: Clarify in the abstract that you are predicting a May-October average (rather than daily predictions).
Line 20: The term “purely parameterized” is used multiple times throughout this manuscript, but I don’t understand what it means or how it is justified. As noted above, the parameterization process seems somewhat ad hoc to me.
Line 23: Suggest clarifying what is meant by a “naïve forecast” here.
Line 44: Models for Lake Erie cyanobacteria blooms (including Bayesian models) predict the maximum bloom size months in advance.
Line 56: Could you explain why “colour” is particularly relevant to water treatment or provide a reference?
Figure 1: Suggest including arrows to show dominant flow directions.
Table 2: Clarify what averaging periods were used.
Line 273: Clarify what normality test was used.
Figure 3, 4, 5: Clarify why only certain features are shown in each figure.
Table 4: The “Feature subset” column is confusing. Use consistent terminology and explain in the caption.
Line 370-371: Revise for clarity.
Line 422: Suggest “wind-related” instead of “related” for clarity.
Line 458: The term “credible” usually refers to the uncertainty in a parameter. It could be good to present actual parameter uncertainties (e.g., credible intervals). Also, I don’t think relationships matching the simple bivariate correlations necessarily makes them “credible” in any sense. For example, see literature on Simpson’s Paradox.
Line 470: Again, I’m not sure using simple bivariate correlations to evaluate a more sophisticated model makes sense.
Table 6: To me, making some numbers bold isn’t effective for highlighting unexpected results. It really depends on which particular pair of numbers is being compared. Also, I wouldn’t describe some of these relationships as a “physical” response.
Line 569: This statement seems too strong and/or requires clarification.
Line 644: This is clearly true (based on the general nature of a GBN), but it wasn’t really explored in this study. I’m not sure why it is a conclusion.
Line 659: This seems like a bit of a stretch. I’m not sure that any “expert” can predict an extreme event ~6 months in advance. Maybe the authors mean something else, but I can’t imagine what.
-
AC2: 'Reply to RC2', Leah Jackson-Blake, 24 Feb 2022
Many thanks to reviewer #2 for a useful set of comments. You have made some very good general points, and I think we can improve the paper by taking them on board.
One of the reviewer’s main concerns was that the paper doesn’t represent a particularly strong contribution. We hope that we can address this by adjusting the way information is presented in the abstract, introduction, discussion and conclusions, and by adding a section on using the GBN to support management. We see the paper’s main contribution as being two-fold:
- We demonstrate a simple, easy-to-use alternative to discrete BNs for water quality modelling. Bayesian Networks (BN) are increasingly popular in environmental modelling, and discrete networks are almost universally used, despite the fact that a number of studies have pointed out their disadvantages when data are continuous. We demonstrate that a simple continuous Gaussian BN (GBN) is easy to develop (our scripts are available; these would be further tidied up and archived in Zenodo during revision), and performs well compared to a standard discrete BN. It is true that we used existing methods and that the GBN performance wasn’t particularly impressive in this case. But it performed as well as/better than the discrete BN, and overall I think that just demonstrating this simple, easy-to-use alternative to discrete BNs for water quality modelling is a useful contribution, given the increasing popularity of BNs. We can try to highlight this point more throughout the text, e.g. by reordering some of the ways information is presented in the abstract, introduction, discussion and conclusions.
- This paper is one of the first to attempt seasonal forecasting of lake water quality. Seasonal forecasting (i.e. where forecasts are produced with lead times of 1-6 months) of streamflow has received a lot of attention in recent decades, whilst seasonal forecasting of water quality is in its infancy (to our knowledge, this would only be the 4th paper on this topic, and 2 of the existing ones relate to flowing waters rather than lakes). To strengthen the paper’s contribution in terms of demonstrating using a BN for seasonal water quality forecasting, we suggest adding in a section to show the lake water quality bulletin we developed, using the GBN prognoses, to support management of the lake.
In the attached response_to_reviewer2 file we provide additional comments in response to individual points raised and suggestions for changes we would make during revisions.
-
RC3: 'Reply on AC2', Anonymous Referee #2, 24 Feb 2022
Quick question: How can this be considered a forecast model if it requires you to input wind speeds 6 months in advance? At best, we can only forecast wind speeds a week or two in advance.
-
AC3: 'Reply on RC3', Leah Jackson-Blake, 24 Feb 2022
Seasonal forecasts for weather variables are available with lead times of up to 6 (or even 9) months, and include wind speed forecasts. Of course you're right that the skill of these forecasts is variable and often low, particularly at longer lead times and outside the tropics (El Nino is the dominant source of seasonal climate predictability, so seasonal climate forecasting skill is often low outside the tropics). Seasonal climate forecasts are also probabilistic and can only give a broad indication of the likely direction of change, e.g. in terms of terciles ("there is a 60% chance that next summer will be windier than normal"). At our study site, we did find that the ECMWF's SEAS5 seasonal climate forecasting system could produce skillful forecasts for a number of weather varriables, including wind speed, in spring (March-May), but skill was lower for the other seasons (Jackson-Blake et al., 2022; https://hess.copernicus.org/preprints/hess-2021-443/). So while you're right that seasonal climate forecasting isn't quite there yet in terms of being able to provide very skilled wind (or temperature or precipitation) forecasts outside the tropics, there are "windows of opportunity" which could be valuable for forecasting water quality, and it is an area of active research which is improving constantly. Of course it ended up being a mute case in this particular study, as neither wind nor precipitation added to the predictive skill of the GBN. If they had, then we would have carried out an extra validation step to look at whether these weather nodes were still worth keeping in the model when seasonal climate forecasts were used to predict seasonal wind and precipitation, instead of observed weather data. This is something which we didn't mention in the current version of the manuscript, but which would probably be worth expanding on a little in any revised version in case others have the same question.
-
RC4: 'Reply on AC3', Anonymous Referee #2, 24 Feb 2022
I think the forecast development narrative is a bit muddled. To say that six-month-ahead wind speed forecasting “isn’t quite there yet” is an understatement. If you are willing to consider 6-month-ahead wind forecasts, then why not also consider 6-month-ahead phosphorus forecasts? The latter is likely much more realistic.
Also, if the GBN can’t provide any measure of credibility of the relationships (e.g., credible intervals for parameters), this is an important limitation that should be noted. I am not a GBN expert, so I can’t provide guidance on how to do this. But it can obviously be done in most linear models (Bayesian or frequentist). Also, probabilistic predictions are easy to obtain from MLR models, so I’d be cautious about over-emphasizing this as an advantage of GBNs.
Overall, I’m not sure if I’ll be able to recommend publication based on the proposed revisions. Of course, I defer to the editor.
-
AC4: 'Reply on RC4', Leah Jackson-Blake, 25 Feb 2022
Relating to the first point in RC4:
- We think it is perfectly reasonable to consider weather variables when developing the forecasting model. Seasonal climate forecasting is a well-established research discipline. While it is clearly not possible to accurately predict wind speeds on a specific day several months in advance, seasonal climate forecasts (like long term climate forecasts) attempt to simulate physically plausible realisations of the future based on current trends and boundary forcings.
- In the context of predicting terciles, seasonal models exhibit significant skill for wind speed variables in certain places and times of year (e.g. Soret et al., 2019, Crespi et al., 2021), including our case study site. It therefore is not an understatement to say wind speed forecasting (for terciles) “isn’t quite there yet”.
- One of the original aims of the study was to see whether the latest seasonal forecasting data products could be used to support water management. As it turned out, we found that including weather variables in the model did not improve predictive performance compared to just using water quality data from the previous season, so this aim became irrelevant and was not mentioned in the paper. However, had weather variables been found to be important predictors of lake water quality, we would have investigated the skill loss associated with replacing observed weather data with seasonal climate hindcasts.
- You suggest we consider 6-month-ahead phosphorus forecasts. The TP node in the GBN is just this - a simple 6-month-ahead model of lake phosphorus concentration. If you instead meant phosphorus forecasts for incoming streamflow, we didn't find any relationship between river TP concentrations or loads and lake water quality. We can mention this.
These points are only briefly touched upon in the current manuscript. If we are invited to revise the paper, we would provide a better background to our motivations for developing the forecasting tool, and to our choice of variables to include in the exploratory feature analysis as it was something which raised some confusion with Reviewer #1 too.
Relating to the second point, we have now calculated 95% confidence intervals on the fitted GBN coefficients. This functionality isn’t offered as standard within e.g. BNLearn (and I doubt it is in other BN packages), so is a bit of a fiddle, but does provide useful extra information. Otherwise, we don’t think we should go too much into the pros and cons of BNs in this paper, as these have been discussed extensively elsewhere. The main aims are instead to demonstrate a simple alternative to discrete BNs for environmental modelling, as well as exploring seasonal water quality forecasting.
-
RC5: 'Reply on AC4', Anonymous Referee #2, 25 Feb 2022
Thank you for the additional notes. I think my last response was a bit hasty. At the same time, it’s somewhat unclear why certain features (covariates) are based on observed data for the period of prediction (which can’t be known at the time of forecast) and other features are based on forecasts of those features. However, I don’t think this is a major sticking point, as the authors can further clarify these issues and their motivation in the manuscript.
I appreciate the authors exploring the credible intervals issue, and I think the proposed demonstration of an example forecast may be helpful. Given that that the GBN shares many of the same features as an MLR (linear relationships, Gaussian error distribution (usually), probabilistic predictions of continuous variables), it would be nice to clarify the potential advantages and disadvantages of the GBN approach. The authors provide some comparison with the discrete BN (Section 4.2), so perhaps something along these lines and with connections to your particular case study.
-
AC5: 'Reply on RC5', Leah Jackson-Blake, 28 Feb 2022
It has become clear through our discussions that we do need to work on further clarifying and justifying the choice of explanatory variables used in the exploratory statistical analysis, the sources of data used for model development, and the sources of data that would be used in any operational forecasting. As well as improving the text, we will have a think about adding a little conceptual figure early in the methods Section to illustrate these points better.
Otherwise, we think the reviewer's suggestion of adding in a comparison of the differences and pros/cons of GBN vs MLR is a good one. Although a comparison of actual predicive performance would also be interesting, we feel like this is outside the scope of the paper, unless the reviewer (and perhaps Editor) feel strongly that it is needed? However, we could certainly add a section comparing the two approaches, probably to the Discussion.
-
RC7: 'Reply on AC5', Anonymous Referee #2, 28 Feb 2022
I agree it isn’t necessary to create an MLR. For one thing, I suspect the predictive skill would be pretty similar to the GBN.
At the same time, it would be nice to elucidate how the GBN could be advantageous relative to more conventional linear statistical models for algal bloom forecasting (while also acknowledging GBN limitations). It's true that BNs have particular features documented in previous literature (e.g., Lines 76-83) but not all of these are unique to BN models, many weren’t demonstrated in this case study, and some might be debatable for a GBN (given the linearity and distributional constraints). Perhaps one important distinction of the GBN is the multivariate structure illustrated in Figure 7. Perhaps the authors could explore and discuss this a bit further in the context of their case study.
Here are a couple of the relevant papers for Lake Erie. One uses an MLR and the other is similar to MLR in some ways (I think). So they might provide good context for this discussion, as well as the discussion of seasonal bloom forecasting, in general.
Obenour, D. R., Gronewold, A. D., Stow, C. A., & Scavia, D. (2014). Using a Bayesian hierarchical model to improve Lake Erie cyanobacteria bloom forecasts. Water Resources Research, 50(10), 7847-7860.
Ho, J. C., & Michalak, A. M. (2017). Phytoplankton blooms in Lake Erie impacted by both long-term and springtime phosphorus loading. Journal of Great Lakes Research, 43(3), 221-228.
Thanks for the interesting discussion. Good luck with your revisions.
-
AC7: 'Reply on RC7', Leah Jackson-Blake, 02 Mar 2022
Thanks for the references, that's useful. When revising our discussion to consider GBN pros/cons/differences compared to MLR, we will make sure to tailor our points to the specifics of our case study, as you are right that some of the more general benefits of BNs may not apply to GBNs, or not be relevant in our study.
Thanks to you too for the discussions, I think that they should lead to a much improved manuscript.
-
AC7: 'Reply on RC7', Leah Jackson-Blake, 02 Mar 2022
-
RC7: 'Reply on AC5', Anonymous Referee #2, 28 Feb 2022
-
AC5: 'Reply on RC5', Leah Jackson-Blake, 28 Feb 2022
-
AC4: 'Reply on RC4', Leah Jackson-Blake, 25 Feb 2022
-
RC4: 'Reply on AC3', Anonymous Referee #2, 24 Feb 2022
-
AC3: 'Reply on RC3', Leah Jackson-Blake, 24 Feb 2022
-
AC2: 'Reply to RC2', Leah Jackson-Blake, 24 Feb 2022
Status: closed
-
RC1: 'Comment on hess-2021-621', Anonymous Referee #1, 21 Jan 2022
Overview
This paper presents a Gaussian Bayesian Network (GBN) for seasonal lake water quality (TP, chl-a, cyanobacteria and colour) forecasting. The GBN was developed and applied to Lake Vansjø in southeast Norway. The GBN was found well-suited for seasonal water quality forecasting and could be parameterised purely using observed data, despite this dataset being small. The forecasting performance of the GBN was assessed using a cross validation scheme, and the performance was also compared to that of a discrete BN (with the same structure) and a naïve forecast model; it was found that the 3 models performed similarly largely due to low interannual variability and high temporal autocorrelation in the study lake.
General comment
Overall, I think this is an interesting study that is very relevant to the HESS special issue. Although the forecasting results with the GBN were considered a mixed success at the study site, I do agree with the authors that the GBN seems to be a sensible and promising approach for water quality forecasting, and I think that by sharing all the code on GitHub the authors have provided a very useful tool for others to use and adapt. I very much enjoyed reading the paper which is both well-written and well-presented, and I believe it can be accepted after some minor revisions.
Below are my comments which I hope the authors will find useful. Lykke til!
Specific comments
- My main concern is related to the discrete BN and the comparison to the GBN. It is interesting that the discrete BN did a mixed job of representing the relationships, however, I don’t understand why this happens and I think this could be elaborated on further. Specific comments in relation to this:
(i) I’m not sure how the method you used to fit the CPTs works, but considering you have a small dataset and that you are using flat priors, I’m surprised that the fitted CPT in Table 6 seems to suggest that the evidence was strong (i.e., most of parent state combinations results in low-high probabilities of around 99%-1% and 95%-5% or vice versa). Intuitively, I would have thought that the probabilities would still be influenced by the flat prior given the small dataset, but the priors have been completely “outweighed” by the data. To me this suggest that there is something odd about the discretisation of the data and/or the target node states.
(ii) I had a brief look at what I believe is your discretised input data files on Github (.. \BayesianNetwork\Data\DataMatrices\Discretized\), and I think these look a bit strange (although I appreciate these may not be the final version). First of all, the ‘colour_prevSummer’ node seems to have been given 3 states (L, M, H) contrary to what is stated in the manuscript. It also looks like the value for ‘chla’ does not always match the value of ‘chla_prevSummer’ the previous year. The same is the case for ‘colour’ and ‘colour_prevSummer’. I would urge the authors to double check these data files and see if this possibly explain (at least partly) the results of the discrete BBN.
(iii) Finally, I wonder if it would not have been better to use expert opinion to reflect the priors in the discrete network before training, especially as you have a small dataset? To me this would seem sensible, and you already use expert opinion to inform the structure of the network. I also wonder whether you could just have discretised your GBN after it was created (in software like Netica and Genie you can specify continuous distributions and then subsequently discretise these distribution) and how the discretised model would then perform? - I’m not sure I fully understand how the leave-one-out cross validation works and I think it would be great if the authors could make this a bit clearer in section 2.7.1. Do you leave one data point (i.e., a year?) out at the time and then fit the GBN to the remaining data and see how well the GBN predicts the target node time-series? Or how well the GBN predicts the data point that was left out? Or something else? I also don’t really understand why the cross validation is stochastic and why it was run a default 20 times.
Minor comments
- Author name: I believe it should be James E. Sample. Alternatively, change JES to JS in author contributions (L670).
- L21: change “wasn’t” to “was not”
- L63-64: maybe worth explaining what polymictic and dimictic lakes are; at least I’m not familiar with these terms.
- Figure 1: where is the outlet from Vanemfjorden? At Moss River?
- L127+: Can you explain briefly why Vanemfjorden with its short residence time is more susceptible to eutrophication and cyano blooms than Storefjorden, and why it does not seem to be related to the major input source from River Hobol?
- L176: Should it be 1998-2013? At least in L179 you seem to suggest NIVA for 2013 as well.
- L188: specify that it is River Hobol
- L192: Change “As the aim” to “The aim”. Alternatively combine the two sentences in L192-195 and remove “therefore” on L194.
- Figure 2: You could consider plotting error bars to give an idea of the variation in the different parameters.
- L227-229: I’m not sure I understand why these features would have to be included as latent variables. Because they are not measured? From Figure 1, it looks like there are monitoring stations in the eastern lake basin (the same as Storefjorden?), so would you not have water quality data from here?
- Table 1 and Table 2: I find it slightly confusing what features are included. Are all the features for the 6-month growing season as well as for the previous winter season (Nov-Apr), i.e., the number of features used for all variables are at least 2x13? Looking at Table 2, and if I understand the caption correctly, it looks like cyano has 8 additional features, so 34 in total (not 33).
- Table 2: Are the features chl-a_prev, cyano chl-a and cyano_prevSummer for the lake?
- L293-300: I think this would be better presented as a table, where you clearly state what is defined as Low and High in the model. The specific comments related to the water quality parameter in question could then be added in a separate column (e.g., that L and H for TP is in fact lower and upper moderate and so on).
- L304+: I don’t follow this part of the discretisation process and why you get unbalanced class sizes. Are the variables still transformed in the discrete version and fairly normal?
- L348+ and Figure 3: Is the relationship between number of calm days and TP negative? To me it looks like the two are positively correlated.
- L355: Are wind speed (winter_wind) and TP(PS) positively correlated?
- Figure 3-6: What are the bell-shaped curves and how were they derived?
- Figure 7: Is TP_prev supposed to be linked to chl-a_prev? If so, should chl-a_prev not have a beta1_TP_prev coefficient?
- L456: Should it not say: “For parentless nodes…”? Some of your nodes are both parent and child nodes (e.g. lake TP is the parent of lake chl-a but the child of TP_prev)
- L526: As you say, this bias in cyano is likely due to the box-cox transformation. Rather than the mean, would it not have been better to use the median (or mode)? Also, did you calculate the mean before or after back-transformation?
- L656: change wasn’t to was not
-
AC1: 'Reply on RC1', Leah Jackson-Blake, 09 Feb 2022
Many thanks for a very useful review, we really appreciate your careful reading of the paper. You have made some very good comments, as well as pointing out some errors and areas where our descriptions were lacking.
In the attached Reviewer1_response.pdf file we provide comments in response to every point raised and suggestions for changes we would make to the text during revisions. Main points in this response include:
Response to major comments:
Firstly, it is great that you think that the data and code might be a useful tool for people. The active repository is a bit of a live (and therefore somewhat messy) workplace, and I will aim to produce a cut-down working version of the code and data used in the paper and archive it in e.g. Zenodo before final publication.
1.(i) We should have provided more detail in the text on the method used for fitting the discrete network. We used the default uniform prior in BNLearn, the very simple so-called ‘K2’ prior, which just adds 1 to the count of every state before counting how often each state of the variable occurs in the data (conditionally on the parent states, where relevant) to estimate the values of the CPDs. The prior doesn’t therefore have much weight compared to the data. However, we could instead have specified an imaginary sample size (iss) >1 (where the iss, or pseudo-counts, are equivalent to having observed iss uniform samples of each variable). Use of a higher iss would have resulted in a smoother (and potentially more realistic) posterior, and I suggest we explore this when revising.
1. (ii) Thanks for digging into the data! This comment can be split into two:
- You’re right, we used 3 states for colour_prevSummer, not the two that we said in the manuscript (and that were used for all other variables), and the text needs correcting. The hope was that in using three classes we would make the most of the extremely strong correlation between colour_prevSummer and colour.
- Yes, the class of e.g. chl-a in 2018 may not be the same as the class of chl-a_prevSummer in 2019 (and likewise for the other lake water quality variables), despite these summarising the same lake growing season. This is because we were tied to using WFD-relevant thresholds to discretize lake TP, chl-a and cyanobacteria for the current season which we aimed to forecast, to make them management-relevant. However, for all other features, and including lake observations from the previous summer, we could instead use a discretization method that would give us better predictive power. In our case, we used regression trees between parent and child variables to pick the thresholds to use in discretization of the parents. We do say this very briefly in Section 2.6, last paragraph, but I suggest we add extra text to clarify and justify the method, including more text to describe the regression-tree based discretization (and its limitations).
1. (iii) Using expert opinion to inform the priors in the discrete network would indeed likely have given better results. However, it would not then have been a fair test compared with the GBN. Your second point here is a good one, and is something we would incorporate in a revised “Discussion”: rather than using a GBN, a discrete network could have been used where continuous distributions were specified and then discretized. This would have resolved the small sample size issue, and I think it should give near identical results to the GBN (assuming normal distributions were assumed), and would not have the same parametric constraints as other continuous distributions could be specified instead. Although it is a slightly clunky solution compared to just developing a GBN, it could be a good alternative for people who use software that does not have GBN capabilities.
2. The description of the cross validation (CV) scheme used needs updating and we will attempt to improve it to make it clearer what was involved. In fact, the method description is currently slightly outdated, which we apologize for. We used k-fold CV, not leave-one-out CV, but with a high value of k (20) so that it approached leave-one-out CV for cyanobacteria (n=23). In k-fold CV the data are randomly assigned to the k subsets, hence the stochastic element.
Response to minor comments:
20. This comment made us reassess our approach to back-transforming the cyanobacteria predictions (which were Box-Cox transformed for fitting the GBN and producing predictions, and then the expected value was back-transformed to the original data scale). Extra reading has made us realise that straight back transforming from a Box-Cox results in a prediction of the median (e.g. Hyndman and Athanasopoulos, 2018, Chapter 3.2). We will explore using a bias-adjusted back transformation to instead calculate the forecasted mean on the original data scale instead. This should reduce the bias in the GBN predictions and may therefore make the GBN perform better, potentially altering some of our results and their interpretation.
References:
Hyndman, Rob J., and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018. https://otexts.com/fpp2/
-
RC6: 'Reply on AC1', Anonymous Referee #1, 28 Feb 2022
Thanks to the authors for the thorough response to my comments. All make sense to me and I'm happy with the proposed changes.
With respect to comment 20 and bias adjustment of box-cox, this was exactly what I was getting at. The following paper might be relevant, which provides a Taylor series expansion for the mean and variance of a "box-cox normal" distribution:
Peter K. Kitanidis & Kuo-Fen Shen (1996): Geostatistical interpolation of chemical concentration. Advances in Water Resources, Vol. 19, No. 6, pp. 369-318, 1996.
This paper also shows how to include the box-cox parameter in the likelihood function for parameter estimation (essentially punishes higher degree of transformation, i.e. lower values of the box cox transformation parameter); although, I'm not sure this latter part is relevant to your study .
Good luck with the revisions
- AC6: 'Reply on RC6', Leah Jackson-Blake, 28 Feb 2022
- My main concern is related to the discrete BN and the comparison to the GBN. It is interesting that the discrete BN did a mixed job of representing the relationships, however, I don’t understand why this happens and I think this could be elaborated on further. Specific comments in relation to this:
-
RC2: 'Comment on hess-2021-621', Anonymous Referee #2, 16 Feb 2022
General comments:
This study presents an application of a Continuous Bayesian Network (CBN) to seasonal (6-month average) algal forecasting in a northern lake. This is likely the first use of CBN for this purpose. In general, the model performs similarly to a traditional (discretized) BN and a naïve model (using the mean from the previous year). It could be a good fit for this special issue, but I do have several concerns, as outlined below.
I’m not really sure that there is a strong contribution, as the CBN does not perform particularly well. Also, the model appears to be based on existing software (an R package), so there isn’t new methods development. If the objective of the study is to provide a thorough demonstration of CBNs for algal bloom modeling, that could potentially be an important contribution. In this case, I’d like to see more demonstrations of how the CBN approach (e.g., Figure 7) can be advantageous for studying a system or supporting management. In my opinion, the current discussion is too focused on skill assessment (e.g., R2), which probably doesn’t do justice to the CBN approach. Also, probabilistic predictions using various linear covariates can also be obtained through multiple linear regression (frequentist or Bayesian), so why use a CBN? I think there are potentially good reasons for using a CBN, but they aren’t compellingly demonstrated in the current manuscript. Also, I’d like to see more discussion of how this effort compares to other CBN (or BN) applications for water quality or environmental sciences, more broadly.
Major comments:
The paper includes a tangential analysis on making predictions at smaller time scales (e.g., Lines 208-215). I recommend removing this material, as it doesn’t seem relevant to the main focus of this paper (no CBN was used). Furthermore, this additional analysis doesn’t provide new insights (that aren’t available through existing phytoplankton literature). It seems a bit “tacked on”. If you do keep this analysis, the data should be presented (as in Figure 2 for the six-month model).
The variable selection process seems ad hoc (Section 3.1.1), making it somewhat hard to follow and likely difficult to reproduce. Some of the explanations seem questionable. For example, the article cites previous literature showing that “windier summers” are relevant, but the CBN uses winds from the previous 6 months (prior to summer), right? I have two general suggestions. First use clear and consistent terminology that clarifies which time periods you are talking about (also use consistent notation across the different figures and tables). Second, drop wind from the 6-month analysis altogether. Much of the text is a somewhat tortuous explanation (at least for this reader) of reasons to include/exclude wind speed, while in reality, the authors readily acknowledge that wind speed is only relevant at smaller time scales (e.g., Lines 443-445: “wind would likely only have an immediate and relatively short-lived effect…”), not ~6 months in advance.
Detail-oriented comments:
Line 11: Clarify in the abstract that you are predicting a May-October average (rather than daily predictions).
Line 20: The term “purely parameterized” is used multiple times throughout this manuscript, but I don’t understand what it means or how it is justified. As noted above, the parameterization process seems somewhat ad hoc to me.
Line 23: Suggest clarifying what is meant by a “naïve forecast” here.
Line 44: Models for Lake Erie cyanobacteria blooms (including Bayesian models) predict the maximum bloom size months in advance.
Line 56: Could you explain why “colour” is particularly relevant to water treatment or provide a reference?
Figure 1: Suggest including arrows to show dominant flow directions.
Table 2: Clarify what averaging periods were used.
Line 273: Clarify what normality test was used.
Figure 3, 4, 5: Clarify why only certain features are shown in each figure.
Table 4: The “Feature subset” column is confusing. Use consistent terminology and explain in the caption.
Line 370-371: Revise for clarity.
Line 422: Suggest “wind-related” instead of “related” for clarity.
Line 458: The term “credible” usually refers to the uncertainty in a parameter. It could be good to present actual parameter uncertainties (e.g., credible intervals). Also, I don’t think relationships matching the simple bivariate correlations necessarily makes them “credible” in any sense. For example, see literature on Simpson’s Paradox.
Line 470: Again, I’m not sure using simple bivariate correlations to evaluate a more sophisticated model makes sense.
Table 6: To me, making some numbers bold isn’t effective for highlighting unexpected results. It really depends on which particular pair of numbers is being compared. Also, I wouldn’t describe some of these relationships as a “physical” response.
Line 569: This statement seems too strong and/or requires clarification.
Line 644: This is clearly true (based on the general nature of a GBN), but it wasn’t really explored in this study. I’m not sure why it is a conclusion.
Line 659: This seems like a bit of a stretch. I’m not sure that any “expert” can predict an extreme event ~6 months in advance. Maybe the authors mean something else, but I can’t imagine what.
-
AC2: 'Reply to RC2', Leah Jackson-Blake, 24 Feb 2022
Many thanks to reviewer #2 for a useful set of comments. You have made some very good general points, and I think we can improve the paper by taking them on board.
One of the reviewer’s main concerns was that the paper doesn’t represent a particularly strong contribution. We hope that we can address this by adjusting the way information is presented in the abstract, introduction, discussion and conclusions, and by adding a section on using the GBN to support management. We see the paper’s main contribution as being two-fold:
- We demonstrate a simple, easy-to-use alternative to discrete BNs for water quality modelling. Bayesian Networks (BN) are increasingly popular in environmental modelling, and discrete networks are almost universally used, despite the fact that a number of studies have pointed out their disadvantages when data are continuous. We demonstrate that a simple continuous Gaussian BN (GBN) is easy to develop (our scripts are available; these would be further tidied up and archived in Zenodo during revision), and performs well compared to a standard discrete BN. It is true that we used existing methods and that the GBN performance wasn’t particularly impressive in this case. But it performed as well as/better than the discrete BN, and overall I think that just demonstrating this simple, easy-to-use alternative to discrete BNs for water quality modelling is a useful contribution, given the increasing popularity of BNs. We can try to highlight this point more throughout the text, e.g. by reordering some of the ways information is presented in the abstract, introduction, discussion and conclusions.
- This paper is one of the first to attempt seasonal forecasting of lake water quality. Seasonal forecasting (i.e. where forecasts are produced with lead times of 1-6 months) of streamflow has received a lot of attention in recent decades, whilst seasonal forecasting of water quality is in its infancy (to our knowledge, this would only be the 4th paper on this topic, and 2 of the existing ones relate to flowing waters rather than lakes). To strengthen the paper’s contribution in terms of demonstrating using a BN for seasonal water quality forecasting, we suggest adding in a section to show the lake water quality bulletin we developed, using the GBN prognoses, to support management of the lake.
In the attached response_to_reviewer2 file we provide additional comments in response to individual points raised and suggestions for changes we would make during revisions.
-
RC3: 'Reply on AC2', Anonymous Referee #2, 24 Feb 2022
Quick question: How can this be considered a forecast model if it requires you to input wind speeds 6 months in advance? At best, we can only forecast wind speeds a week or two in advance.
-
AC3: 'Reply on RC3', Leah Jackson-Blake, 24 Feb 2022
Seasonal forecasts for weather variables are available with lead times of up to 6 (or even 9) months, and include wind speed forecasts. Of course you're right that the skill of these forecasts is variable and often low, particularly at longer lead times and outside the tropics (El Nino is the dominant source of seasonal climate predictability, so seasonal climate forecasting skill is often low outside the tropics). Seasonal climate forecasts are also probabilistic and can only give a broad indication of the likely direction of change, e.g. in terms of terciles ("there is a 60% chance that next summer will be windier than normal"). At our study site, we did find that the ECMWF's SEAS5 seasonal climate forecasting system could produce skillful forecasts for a number of weather varriables, including wind speed, in spring (March-May), but skill was lower for the other seasons (Jackson-Blake et al., 2022; https://hess.copernicus.org/preprints/hess-2021-443/). So while you're right that seasonal climate forecasting isn't quite there yet in terms of being able to provide very skilled wind (or temperature or precipitation) forecasts outside the tropics, there are "windows of opportunity" which could be valuable for forecasting water quality, and it is an area of active research which is improving constantly. Of course it ended up being a mute case in this particular study, as neither wind nor precipitation added to the predictive skill of the GBN. If they had, then we would have carried out an extra validation step to look at whether these weather nodes were still worth keeping in the model when seasonal climate forecasts were used to predict seasonal wind and precipitation, instead of observed weather data. This is something which we didn't mention in the current version of the manuscript, but which would probably be worth expanding on a little in any revised version in case others have the same question.
-
RC4: 'Reply on AC3', Anonymous Referee #2, 24 Feb 2022
I think the forecast development narrative is a bit muddled. To say that six-month-ahead wind speed forecasting “isn’t quite there yet” is an understatement. If you are willing to consider 6-month-ahead wind forecasts, then why not also consider 6-month-ahead phosphorus forecasts? The latter is likely much more realistic.
Also, if the GBN can’t provide any measure of credibility of the relationships (e.g., credible intervals for parameters), this is an important limitation that should be noted. I am not a GBN expert, so I can’t provide guidance on how to do this. But it can obviously be done in most linear models (Bayesian or frequentist). Also, probabilistic predictions are easy to obtain from MLR models, so I’d be cautious about over-emphasizing this as an advantage of GBNs.
Overall, I’m not sure if I’ll be able to recommend publication based on the proposed revisions. Of course, I defer to the editor.
-
AC4: 'Reply on RC4', Leah Jackson-Blake, 25 Feb 2022
Relating to the first point in RC4:
- We think it is perfectly reasonable to consider weather variables when developing the forecasting model. Seasonal climate forecasting is a well-established research discipline. While it is clearly not possible to accurately predict wind speeds on a specific day several months in advance, seasonal climate forecasts (like long term climate forecasts) attempt to simulate physically plausible realisations of the future based on current trends and boundary forcings.
- In the context of predicting terciles, seasonal models exhibit significant skill for wind speed variables in certain places and times of year (e.g. Soret et al., 2019, Crespi et al., 2021), including our case study site. It therefore is not an understatement to say wind speed forecasting (for terciles) “isn’t quite there yet”.
- One of the original aims of the study was to see whether the latest seasonal forecasting data products could be used to support water management. As it turned out, we found that including weather variables in the model did not improve predictive performance compared to just using water quality data from the previous season, so this aim became irrelevant and was not mentioned in the paper. However, had weather variables been found to be important predictors of lake water quality, we would have investigated the skill loss associated with replacing observed weather data with seasonal climate hindcasts.
- You suggest we consider 6-month-ahead phosphorus forecasts. The TP node in the GBN is just this - a simple 6-month-ahead model of lake phosphorus concentration. If you instead meant phosphorus forecasts for incoming streamflow, we didn't find any relationship between river TP concentrations or loads and lake water quality. We can mention this.
These points are only briefly touched upon in the current manuscript. If we are invited to revise the paper, we would provide a better background to our motivations for developing the forecasting tool, and to our choice of variables to include in the exploratory feature analysis as it was something which raised some confusion with Reviewer #1 too.
Relating to the second point, we have now calculated 95% confidence intervals on the fitted GBN coefficients. This functionality isn’t offered as standard within e.g. BNLearn (and I doubt it is in other BN packages), so is a bit of a fiddle, but does provide useful extra information. Otherwise, we don’t think we should go too much into the pros and cons of BNs in this paper, as these have been discussed extensively elsewhere. The main aims are instead to demonstrate a simple alternative to discrete BNs for environmental modelling, as well as exploring seasonal water quality forecasting.
-
RC5: 'Reply on AC4', Anonymous Referee #2, 25 Feb 2022
Thank you for the additional notes. I think my last response was a bit hasty. At the same time, it’s somewhat unclear why certain features (covariates) are based on observed data for the period of prediction (which can’t be known at the time of forecast) and other features are based on forecasts of those features. However, I don’t think this is a major sticking point, as the authors can further clarify these issues and their motivation in the manuscript.
I appreciate the authors exploring the credible intervals issue, and I think the proposed demonstration of an example forecast may be helpful. Given that that the GBN shares many of the same features as an MLR (linear relationships, Gaussian error distribution (usually), probabilistic predictions of continuous variables), it would be nice to clarify the potential advantages and disadvantages of the GBN approach. The authors provide some comparison with the discrete BN (Section 4.2), so perhaps something along these lines and with connections to your particular case study.
-
AC5: 'Reply on RC5', Leah Jackson-Blake, 28 Feb 2022
It has become clear through our discussions that we do need to work on further clarifying and justifying the choice of explanatory variables used in the exploratory statistical analysis, the sources of data used for model development, and the sources of data that would be used in any operational forecasting. As well as improving the text, we will have a think about adding a little conceptual figure early in the methods Section to illustrate these points better.
Otherwise, we think the reviewer's suggestion of adding in a comparison of the differences and pros/cons of GBN vs MLR is a good one. Although a comparison of actual predicive performance would also be interesting, we feel like this is outside the scope of the paper, unless the reviewer (and perhaps Editor) feel strongly that it is needed? However, we could certainly add a section comparing the two approaches, probably to the Discussion.
-
RC7: 'Reply on AC5', Anonymous Referee #2, 28 Feb 2022
I agree it isn’t necessary to create an MLR. For one thing, I suspect the predictive skill would be pretty similar to the GBN.
At the same time, it would be nice to elucidate how the GBN could be advantageous relative to more conventional linear statistical models for algal bloom forecasting (while also acknowledging GBN limitations). It's true that BNs have particular features documented in previous literature (e.g., Lines 76-83) but not all of these are unique to BN models, many weren’t demonstrated in this case study, and some might be debatable for a GBN (given the linearity and distributional constraints). Perhaps one important distinction of the GBN is the multivariate structure illustrated in Figure 7. Perhaps the authors could explore and discuss this a bit further in the context of their case study.
Here are a couple of the relevant papers for Lake Erie. One uses an MLR and the other is similar to MLR in some ways (I think). So they might provide good context for this discussion, as well as the discussion of seasonal bloom forecasting, in general.
Obenour, D. R., Gronewold, A. D., Stow, C. A., & Scavia, D. (2014). Using a Bayesian hierarchical model to improve Lake Erie cyanobacteria bloom forecasts. Water Resources Research, 50(10), 7847-7860.
Ho, J. C., & Michalak, A. M. (2017). Phytoplankton blooms in Lake Erie impacted by both long-term and springtime phosphorus loading. Journal of Great Lakes Research, 43(3), 221-228.
Thanks for the interesting discussion. Good luck with your revisions.
-
AC7: 'Reply on RC7', Leah Jackson-Blake, 02 Mar 2022
Thanks for the references, that's useful. When revising our discussion to consider GBN pros/cons/differences compared to MLR, we will make sure to tailor our points to the specifics of our case study, as you are right that some of the more general benefits of BNs may not apply to GBNs, or not be relevant in our study.
Thanks to you too for the discussions, I think that they should lead to a much improved manuscript.
-
AC7: 'Reply on RC7', Leah Jackson-Blake, 02 Mar 2022
-
RC7: 'Reply on AC5', Anonymous Referee #2, 28 Feb 2022
-
AC5: 'Reply on RC5', Leah Jackson-Blake, 28 Feb 2022
-
AC4: 'Reply on RC4', Leah Jackson-Blake, 25 Feb 2022
-
RC4: 'Reply on AC3', Anonymous Referee #2, 24 Feb 2022
-
AC3: 'Reply on RC3', Leah Jackson-Blake, 24 Feb 2022
-
AC2: 'Reply to RC2', Leah Jackson-Blake, 24 Feb 2022
Leah Jackson-Blake et al.
Leah Jackson-Blake et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 648 | 201 | 30 | 879 | 7 | 7 |
- HTML: 648
- PDF: 201
- XML: 30
- Total: 879
- BibTeX: 7
- EndNote: 7
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1