General comments:

The paper is very well written and very detailed. The topic of the study is certainly of great interest to the forecasting community. The combination of a hydrological uncertainty processor (MCP) and EMOS with the help of a Kalman filter approach is quite novel and attractive. The discussion of the results is, however, a bit too long and could be summarized more concisely focusing on floods aspects. Therefore, I suggest that it is worth publishing after some minor revision.

There are many EFAS paper available now and all describe the EFAS system. This could be shortened and only the differences to the operational settings of EFAS should be explained, in particular how the reforecasts are used.  The biggest challenge in using reforecasts is the reduced number of ensemble members (11 instead of 51). This small number of members causes difficulties in computing the CRPS for making a fair comparison with a CRPS derived from the PDFs of the post-processed forecasts (see for example Zamo and Naveau, 2018). However, this problem is not mentioned and the presented results of the CRPSS should be treated with care.

A most recent paper from Skoven, et al. (2021) has a similar topic about evaluating the post-processing methods for EFAS (EMOS and the application of transformations like NQT). Therefore, the differences and novelty of this study should be stressed clearly and discussed in more detail. One difference is, apart from using the reforecasts and MCP, that the EMOS correction parameters are lead-time invariant. This could be stressed more clearly right from the beginning. For me, this lead-time invariance is a big drawback of the proposed method and rather problematic for deriving the total predictive uncertainty, which forces the Kalman filter sometimes to give more wait to the hydrological uncertainties.

Although the analysis of the different aspects like catchment size, elevation, regulation, length of period, are very interesting, it maybe could be shortened focusing on floods, which is the main topic of EFAS. Also, the detailed analysis of the Nash-Sutcliffe (KGE) is maybe too long, since the results of the post-processing methods are probabilistic and the KGE reduces the information content to the mean (median) of the Ensembles (or PDF). Therefore, I would suggest that the emphasis of the verification should be the CRPS.

Specific comments

Page 5: Highlighting the differences between the operational setting of EFAS and the setting used in this study should be sufficient. More details can be found in many other papers. However, the calibration period of the LISFLOOD model is missing. Is there an overlap between the period for calibrating the parameters of the hydrological model and the historical period p for the off-line calibration?   

In Figure 1, the index of the parameters µ and Σ is ψ, whereas in the caption you use the index Φ.

You mention several times (e.g. line 172, page 7) that the minimum for the off-line calibration is 2 years. However, for the fitting of the GPD you use 1000 values (page 9). So you will need more than 2 years?   

I would suggest to include a list of nomenclature, so you avoid repetitive descriptions like the tilde for the physical space (line 182, 204, 220, 340) and the definition of the timestep notation introduced on page 7 (lines 176-177).

On page 9, line 235, you write that the location parameter a is used for defining the breakpoint, but in Figure 2 the shape parameter c is used as breakpoint.

On page 9, line 251, you write about consistence between the 2 distribution. What does it mean? How do you check this?

On page 10, line 257 the concentrated likelihood method is mentioned without any further explanation what this method does. Maybe some more details would be helpful. Also, it is not clear for me how the GPD is weighted (line 261-262) .

Whereas the description of the linear approximation (page 11 – line 270 – 280) is maybe not necessary.

On page 13, it is not clear for me why you have observations (line 336) and water balance (line 337) for the period until t, but the forecasts (line 339) until t-1 ?

Line 349: …using a MCP method …

Page 14, line 372: in the recent period …

Page 16, line 411: you write that a set of forecasts are used estimate the two spread correction parameters. How did you choose the size of these sets of forecasts ?

 In the Figure 3 you write CRPS besides the legend bar, but it should be CRPSS?

On page 19, line 483 you write that only 11 reforecasts are available (I suppose this 11 comes from 40 days/7day x 2). The number 11 could be misleading, since it happens to coincide with the number of members mentioned in the next sentence (line 485).  Why do you fix it to 40 days? Since there is this a discrepancy between the operational setting and this analysis anyhow, you could set q to a longer period to include more reforecasts (e.g q=70 ~ 20 reforecasts).

At line 486 I don’t understand why the mean discharge value is predicted for the previous 6 hours?

The difference between the raw and the post-processed forecasts in Fig. 11a (mentioned on page 35, line 862) is very difficult to see and almost not visible.

Line 880: …uncertainties show a small increase

On page 41 the paragraph from line 1004 – 1010 can be removed

Line 1021  ..greater than..

I have some doubts about your suggestion that very short periods are sufficient (line 1045): the chance that such a short period will show the variability of the discharge needed for applying the NQT is rather small and the fitting of the GPD almost impossible. Consequently the back-transformation of the variables from the Normal space will always produce poor and very unreliable results for floods.

The citation of Coccia (line 1135) is incomplete. Also, the term “Multi-Temporal” in combination with the MCP (MT-MCP) is mentioned only in line 153-154 and in the conclusions (line 1054), but is not explained.

Zamo, M., Naveau, P. Estimation of the Continuous Ranked Probability Score with Limited Information and Applications to Ensemble Weather Forecasts. Math Geosci 50, 209–234 (2018). https://doi.org/10.1007/s11004-017-9709-7)

Skøien, J. O., Bogner, K., Salamon, P., & Wetterhall, F. (2021). On the Implementation of Postprocessing of Runoff Forecast Ensembles, Journal of Hydrometeorology, 22(10), 2731-2749

This paper describes a novel post-processing method applied to the EFAS forecasts, assesses the improvements to forecasts realised by post-processing to a large number of catchments and investigates factors that influence the performance of the post-processor. The paper is very well structured and written, and the topic is of considerable interest to the forecasting researchers and practioners using the EFAS forecasts. 

The paper is comprehensive covering the post-processing method itself, the improvements across the EFAS domain and the factors influencing the forecast performance, which necessitates a lengthy manuscript. All aspects presented are of interest, however I do wonder whether the paper could be separated into two more focussed manuscipts, perhaps one focussing on the novel aspects of the post-processing method and validating its assumptions, and a second on evaluation the benefits and investigating factors that influence its performance.

More specific comments:

The sample covariance matrix is used to characterise the joint distribution of the historic observations and water balance simulations, equantion 7. There are a potential issues that may be encountered using this approach and it would be good understand whether special treatments have been needed to overcome these. Specific issues that come to mind include: (i) The covariance matrix is computed over a set of historic observations and is likely to have inflated, or spurious, correlations over long lags if the seasonal cycle of streamflow is not considered. These inflated/spurious correlations are likely to lead to inflated variances of conditional predictions. (ii) The authors indicate that there missing (and possibly zero-valued) observations that are used in the estimation of the covariance matrices. For large sample covariance matrices such as those estimated in this study, missing observations can lead covariance matrices that are not positive definite. Have any issues been identified and any special treatment been implemnted to deal with  

The KGE analysis is performed using the median as a point estimate of the forecast ensemble. The results obtained for the post-processed forecasts, particularly the bias ratios and variability ratios of less than one at long lead times, are not unexpected as the variance of the forecast median will be considerable more damped that the mean. The forecast mean is likely to be a better choice as the point estimate of the forecast ensemble. Some theoretical justification of the use of the ensemble mean with measures of squared error can be found in Gneiting (2011).

Analysis of forecasts for extreme events such as floods requires careful design to ensure that the performance evaluation is not biased (Lerch, 2017). In this paper, the analysis of peak timing is conditioned on observations exceeding a threshold (90th percentile discharge threshold) within the forecast period, and is likely to result in a biased evaluation of forecasts. A more rigourous approach would be to select the events based on forecasts exceeding the threshold. I also believe that rather than evaluating the timing of the peak in the forecast median, which doesn't correspond to the peak in any individual hydrograph, a more representative point estimate of the forecast timing error would be to compare the median (or mean) time to peak across all ensemble members to the timing of the observed peak. 

line 373 - values in the recent perion should be "values in recent period"

Line 825 - CRPS calculated on deterministic forecasts is equivalent to the absolute error not the square absolute error.

Figures - The size of multi-panel figures (e.g. Figure 9, 12) could be increased to better illustrate the detail,

References:

Tilmann Gneiting (2011) Making and Evaluating Point Forecasts, Journal of the American Statistical Association, 106:494, 746-762, DOI: 10.1198/jasa.2011.r10138

Sebastian Lerch. Thordis L. Thorarinsdottir. Francesco Ravazzolo. Tilmann Gneiting. "Forecaster’s Dilemma: Extreme Events and Forecast Evaluation." Statist. Sci. 32 (1) 106 - 127, February 2017. https://doi.org/10.1214/16-STS588