Assessing multivariate bias corrections of climate simulations on various impact models under climate change

Allard, Denis; Vrac, Mathieu; François, Bastien; García de Cortázar-Atauri, Iñaki

doi:https://doi.org/10.5194/hess-2024-102

Preprints

https://doi.org/10.5194/hess-2024-102

Preprints

28 Aug 2024

| 28 Aug 2024

Status: a revised version of this preprint is currently under review for the journal HESS.

Assessing multivariate bias corrections of climate simulations on various impact models under climate change

Denis Allard, Mathieu Vrac, Bastien François, and Iñaki García de Cortázar-Atauri

Abstract. Atmospheric variables simulated from climate models often present biases relative to the same variables calculated by reanalysis in the past. In order to use these models to assess the impact of climate change on processes of interest, it is necessary to correct these biases. Currently, the bias correction methods used operationally correct one-dimensional time series and are therefore applied separately, physical variable by physical variable and site by site. Multivariate bias correction methods have been developed to better take into account dependencies between variables and in space. Although the performance of multivariate bias correction methods for adjusting the statistical properties of simulated climate variables has already been evaluated, their effectiveness for different impact models has been little investigated. In this work, we propose a comparison between two multivariate bias correction methods (R2D2 and dOTC) in three different configurations (intervariable, spatial and spatial-intervariable) and a univariate correction (CDF-t) through several highly multivariate impact models (phenological stage, reference evapotranspiration, soil water content, fire weather index) integrating the weather conditions over a whole season. Our results show that CDF-t does a fair job in most situations and that there is no single best MBC method. The performances of multivariate bias correction methods depend both on some characteristics of the studied process and on the configuration of the chosen bias correction method. When all characteristics are important (multivariate, time cumulative and spatial) it is found that dOTC in its spatial-intervariable configuration brings improvements in most cases and no significant improvement in some rare cases. We did not find any multivariate cases where the spatial-intervariable configuration for dOTC performs less well than CDF-t.

Received: 05 Apr 2024 – Discussion started: 28 Aug 2024

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Denis Allard, Mathieu Vrac, Bastien François, and Iñaki García de Cortázar-Atauri

Status: final response (author comments only)

RC1:
'Comment on hess-2024-102', Anonymous Referee #1, 01 Oct 2024
Reviewer Comments on “Assessing Multivariate Bias Corrections of Climate Simulations on Various Impact Models Under Climate Change”
The manuscript is extensive and well-written, providing a thorough evaluation of two multivariate bias correction (MBC) methods—R2D2 (Vrac, 2018) and dOTC (Robin et al., 2019)—to adjust the inter-variable and spatial dependence structures of five physical variables used as inputs for various process models. For each MBC method, three different configurations were considered (inter-variable, spatial, and spatial-intervariable) to disentangle the relative effects of the various dependence structures. A univariate bias correction method (CDF-t, Michelangeli et al., 2009) was also included to assess the potential added value of MBC methods.
The authors found that CDF-t performs adequately in most situations and that there is no single best MBC method. The performance of multivariate bias correction methods depends on the characteristics of the studied process and the configuration of the chosen bias correction method. When all characteristics are important (multivariate, time cumulative, and spatial), it is found that dOTC in its spatial-intervariable configuration brings improvements in most cases and no significant improvement in some rare cases.
Major Concern:
My main concern is related to the use of quantile mapping as a basis for the MBC methods. It is argued that under future climate scenarios, both extreme dry and wet events are projected to increase, potentially leading to unprecedented extreme conditions. If basic quantile mapping is used as a basis of the bias correction method, there is a risk of losing these extreme events in future projections due to its limitations in extrapolating beyond observed extremes. How do the authors address this issue in their methodology? Have they considered using advanced bias correction techniques that better preserve or model extreme events under climate change?
Additional Comments:
Abstract:

The abbreviations R2D2, dOTC, CDF-t, and MBC are used without first providing their full terms. It would enhance clarity to define these abbreviations upon their first mention in the abstract.
Introduction:

The introduction would benefit from a more detailed and systematic literature review of the development of multivariate bias correction methods. Discussing the advantages and disadvantages of existing methods would provide valuable context and help highlight the contribution of this study.

Figure 1:

It would be helpful if the subtitles provided more detailed information about each panel. Additionally, labeling the subfigures with identifiers like (a), (b), (c) or numbers like 1, 2, 3 would improve readability and make it easier to reference specific parts of the figure in the text.

Line 180:

Including a simple formula to show how the Soil Water Content (SWC) is calculated would enhance the reader's understanding of the methodology and the variables involved.

Line 187:

Please discuss the impact of the simplicity of the model or assumptions mentioned here. Elaborating on how this simplicity might affect the results or interpretations would strengthen the credibility of the study.

Line 216:

Under climate change, inter-variable correlations might change over time. How does the methodology account for potential changes in inter-variable correlations in future climate scenarios? Addressing this point would clarify the robustness of the bias correction methods when applied under changing climatic conditions.

Citation: https://doi.org/10.5194/hess-2024-102-RC1

AC1: 'Reply on RC1', Denis ALLARD, 04 Jul 2025

First of all, we express our gratitude to the referee for the valuable comments and feedback. We made efforts to incorporate all of their suggestions, and we believe that the manuscript has been enhanced as a result.
Below, we provide a detailed response to the reviewers' comments, with our responses in bold font.

1. The introduction would benefit from a more detailed and systematic literature review of the development of multivariate bias correction methods. Discussing the advantages and disadvantages of existing methods would provide valuable context and help highlight the contribution of this study.
We agree that we have provided a rather rapid overview of existing Multivariate Bias Correction (MBC) methods. However, it is not possible in this paper to provide a detailed description of all methods in this paper, which is not a review paper. We have thus decided to expand the paragraph presenting the MBC methods (lines 58-69) in the original manuscript, which will now read:
“To adjust biases in multivariate dependencies, some multivariate bias correction (MBC) methods have been recently developed in the literature. The objective of MBC is fundamentally the same as that of univariate BC: transforming climate model simulations so that selected statistical features match those of a reference data set over the calibration period. The difference with univariate BCs lies in the fact that statistical features are not only univariate, but also multivariate such as inter-variable correlations or spatial copula structure. MBC methods can be grouped into three categories based on how they adjust climate simulations, (Vrac (2018); Robin et al. (2019); François et al. (2020): 1) most methods belong to the "marginal/dependence'' category that gathers MBCs adjusting separately univariate distributions and dependence properties. For precipitation outputs of RCMs, Bárdossy and Pegram (2012) proposed a correction of the spatial structure using matrix recorrelation or sequential correlation based on copulas. For correcting spatiotemporal biases for multiple variables, Cannon (2018) proposed an image processing technique for multivariate and spatial bias correction. Sharma (2019) and Guo et al. (2019) proposed a modification of the ranks (obtained from the empirical quantiles) based on resampling for the former and shuffling for the latter. In the same spirit, but based on CDF-t for the univariate correction instead a simple quantile mapping, Vrac (2018) and Vrac and Thao (2020) proposed a "Rank Resampling for Distributions and Dependencies" bias correction (R2D2), which will be detailed later. In principle, these approaches are not limited in terms of the number of variables, grid points in space, and the time scale considered. Nahar et al. (2018) proposed an independent component analysis approach to correct biases at multiple locations conjointly, but this approach must be applied independently to each climate variable. Nguyen et al. (2019) considered a bias correction approach for time series using the spectrum in the frequency domain, applicable to multivariate time series or to several grid cells. François et al. (2021) proposed a Cycle-GAN, cycle-consistent adversarial network for the adjustment of spatial dependence structures of climate model projections. 2) The “successive conditional” correction approach, that adjusts the different simulated variables successively and conditionally on the previously adjusted ones (e.g., Piani and Haerter, 2012; Dekens et al., 2017). 3) The “all-in-one”' category that consists of MBCs simultaneously adjusting univariate and multivariate properties of climate simulations. Pan et al. (2021) proposed to learn an adversarial neural network for spatially coherent corrections of daily precipitations, while Robin et al. (2019) use optimal transport theory for multivariate and spatially coherent bias corrections."

2. It would be helpful if the subtitles in Figure 1 provided more detailed information about each panel. Additionally, labeling the subfigures with identifiers like (a), (b), (c) or numbers like 1, 2, 3 would improve readability and make it easier to reference specific parts of the figure in the text.
Done, thanks.
3. Line 180 Including a simple formula to show how the Soil Water Content (SWC) is calculated would enhance the reader's understanding of the methodology and the variables involved.
We added the following sentence describing how SWC is computed (now line 198) “by computing SWCt = SWCt-1 + (prt - k.ET0t )/ d, where d is the depth of the soil and k is a constant depending on the considered crop”.
4. Line 187: Please discuss the impact of the simplicity of the model or assumptions mentioned here. Elaborating on how this simplicity might affect the results or interpretations would strengthen the credibility of the study.
Multiplying the type of soils with possibly different depth and/or different soil water capacities would have significantly increased the experiment without significantly changing our results. We ran sensitivity tests in some settings. Even though some differences were visible in the SWC values, the conclusions in terms of bias correction methods remained unchanged (not shown). Actually, this result also holds when considering different wheat varieties for example. In order to keep the experimental design at a manageable size, we decided to run the full study for a single soil type and a single variety per crop. We thus added the following sentence: “Early sensitivity tests have shown that when changing these parameters, even though some differences were visible in the SWC values, the conclusions in terms of bias correction methods that will be reported later remained unchanged (not shown). In order to keep the experimental design at a manageable size, we decided to run the full study for a single soil type. This choice also avoids problems of soil heterogeneity in each region and allows us to catch as much climate effect as possible.”

5. Line 216: Under climate change, inter-variable correlations might change over time. How does the methodology account for potential changes in inter-variable correlations in future climate scenarios? Addressing this point would clarify the robustness of the bias correction methods when applied under changing climatic
With dOTC, a transport map between the multivariate density on the model and that on the reference on the calibration period is estimated. This transport map is then used to correct the model outputs in the projection period. The underlying hypothesis is that this transport map remains identical in the future, but it does not assume that the inter-variable correlations do not change in the future. Indeed, dOTC accounts for the changes in the multivariate distribution (and thus in the inter-variable correlations) and incorporates them into the corrections (Robin et al., 2019). Regarding R2D2, the marginal distributions can change in the future while the copula is considered stationary in time, and identical to that in the reference. This might seem a strong assumption, but Vrac et al. (2023) showed that this was a conservative hypothesis preventing climate models from exhibiting over-varying inter-variable correlations, not corresponding to actual (historical) evolutions (Vrac et al., 2022). In conclusion, assuming a stationary copula corresponding to reference data is a safe choice.
We added the following text at the end of the presentation of the R2D2 method (line 239).
“While this may appear to be a strong assumption, Vrac et al. (2023) showed that it serves as a conservative hypothesis. This prevents climate models from exhibiting over-varying inter-variable correlations that do not correspond to actual (historical) evolutions (Vrac et al., 2022). Assuming a stationary copula corresponding to reference data is thus a safe choice.”
Additional references:
Vrac, M., Thao, S., and Yiou, P.: Should multivariate bias corrections of climate simulations account for changes of rank correlation over time?, Journal of Geophysical Research: Atmospheres, 127, e2022JD036 562, 2022.
Vrac, M., Thao, S., and Yiou, P. (2023) : Changes in temperature–precipitation correlations over Europe: are climate models reliable?, Climate Dynamics, 60, 2713–2733, 2023.

Citation: https://doi.org/10.5194/hess-2024-102-AC1

RC2: 'Comment on hess-2024-102', Stefano Galmarini, 11 Jun 2025

The paper present a comparative analysis of multivariate bias correction methods, that is performed with great care, excellent methodological set up and wealth of evidence in support of the conclusions. As far as I am concerned the manuscript can proceed toward publication almost as it is, being also extremely well written and clear.

My 'almost' is motivated by two requests of clarification and eventually of modification.

1- the domain area selection: while one can appreciate the selection of highly diverse sub-region of France, one can also wonder why the areas are all comparable in size and what would happen to the analysis should these areas be much larger in size. Was the choice of these sizes motivated by computational constraints or the authors see a problem in the methods performance should the domain be larger? An elaboration on this aspects in the conclusions or the case set up, I assume, is due.

2- From the Conclusions section:

''Galmarini et al. (2024) considered a total of 12 crop models, which are highly multivariate and integrative in time,

but the spatial dimension was not considered at all. They found that R2D2 (I.R2D2 configuration) was among the best

performing method, which is in contradiction with our findings showing that I.R2D2 and SI.R2D2 do not adjust better

than CDF-t for SWC and FWI. The exact reason for this discrepancy should be explored in future work.''

While the first part of this conclusion is indisputable, the second part merits a more nuanced language and detailed analysis. In fact in Table 3 of Galmarini et al. (2024) it clearly appears that while multivariate methods outperform univariate ones, CDFt outperforms all its siblings but also scores in the first second and third error rankings frequencies comparable to the other multivariate methods and also R2D2 (ranking 2 and 3). Therefore for the specific comparison with CDFt, Galmarini et al. (2024) conclusion is not that far from those of the current manuscript. Please modify accordingly, if you agree. We agree that CDFt appears to be an outstanding univariate method whose properties should be indeed further investigated.

As for the rest I have nothing else to object and while waiting for the response of the authors I wish to congratulate them on this nice piece of research.

Citation: https://doi.org/10.5194/hess-2024-102-RC2

AC2: 'Reply on RC2', Denis ALLARD, 04 Jul 2025

First of all, we express our gratitude to the referee for his general positive appraisal and for valuable comments. We addressed the two points raised by the referee. Below, we provide a detailed response to the reviewers' comments, with the responses in bold font.
1- The domain area selection: while one can appreciate the selection of highly diverse sub-region of France, one can also wonder why the areas are all comparable in size and what would happen to the analysis should these areas be much larger in size. Was the choice of these sizes motivated by computational constraints or the authors see a problem in the methods performance should the domain be larger? An elaboration on this aspects in the conclusions or the case set up, I assume, is due.
This is indeed a very good point. We added the following in the discussion (now line 617):
“In this study, we have considered three areas of rather limited size containing between 259 and 337 grid points. CDF-t and Intervar settings are insensitive to the size of the domain. R2D2 can in principle handle very large vectors, hence very large domains, but we have seen that the S and SI settings are not to be recommended when spatial coherence is important. On multivariate processes with time integration for which the spatial structure matters, SI.dOTC brings improvement in all tested situations, i.e. in areas ranging from 4000 to 5000 km$^2$. Further tests are necessary to assess SI.dOTC on a larger number of grid points.”
2- From the Conclusions section:
''Galmarini et al. (2024) considered a total of 12 crop models, which are highly multivariate and integrative in time, but the spatial dimension was not considered at all. They found that R2D2 (I.R2D2 configuration) was among the best performing method, which is in contradiction with our findings showing that I.R2D2 and SI.R2D2 do not adjust better than CDF-t for SWC and FWI. The exact reason for this discrepancy should be explored in future work.''
While the first part of this conclusion is indisputable, the second part merits a more nuanced language and detailed analysis. In fact in Table 3 of Galmarini et al. (2024) it clearly appears that while multivariate methods outperform univariate ones, CDFt outperforms all its siblings but also scores in the first second and third error rankings frequencies comparable to the other multivariate methods and also R2D2 (ranking 2 and 3). Therefore for the specific comparison with CDFt, Galmarini et al. (2024) conclusion is not that far from those of the current manuscript. Please modify accordingly, if you agree. We agree that CDFt appears to be an outstanding univariate method whose properties should be indeed further investigated.
Thank you for your comment. This is absolutely correct, we thus changed our text which now reads (lines 596 - 604).
''Galmarini et al. (2024) considered a total of 12 crop models, which are highly multivariate and integrative in time, but the spatial dimension was not considered at all. They found that R2D2 (I.R2D2 configuration) was among the best performing methods, second to CDF-t. Our results complement this finding with new results stemming from spatial settings. We also found that CDF-t is among the methods performing best, but that R2D2 underperforms in some situations. It must be remembered that R2D2 leaves the temporal dynamics of the pivot variable unchanged. Here, the pivot variable was temperature, and it was also the case in Galmarini et al. (2024). In this case, processes that are mostly driven by temperature will perform fairly well, as reported in Galmarini et al. (2024). When variables other than the pivot are important (i.e. for SWC and FWI which depend highly on precipitation), we found that I.R2D2 and SI.R2D2 do not adjust better than CDF-t for SWC and FWI.”

Citation: https://doi.org/10.5194/hess-2024-102-AC2

Denis Allard, Mathieu Vrac, Bastien François, and Iñaki García de Cortázar-Atauri

Viewed

Total article views: 898 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
565	140	193	898	24	34

HTML: 565
PDF: 140
XML: 193
Total: 898
BibTeX: 24
EndNote: 34

Views and downloads (calculated since 28 Aug 2024)

Month	HTML	PDF	XML	Total
Aug 2024	121	38	8	167
Sep 2024	95	25	4	124
Oct 2024	43	5	28	76
Nov 2024	36	3	46	85
Dec 2024	29	5	54	88
Jan 2025	28	9	39	76
Feb 2025	30	6	2	38
Mar 2025	31	10	3	44
Apr 2025	30	10	1	41
May 2025	36	8	1	45
Jun 2025	57	10	3	70
Jul 2025	29	11	4	44

Cumulative views and downloads (calculated since 28 Aug 2024)

Month	HTML	PDF	XML	Total
Aug 2024	121	38	8	167
Sep 2024	95	25	4	124
Oct 2024	43	5	28	76
Nov 2024	36	3	46	85
Dec 2024	29	5	54	88
Jan 2025	28	9	39	76
Feb 2025	30	6	2	38
Mar 2025	31	10	3	44
Apr 2025	30	10	1	41
May 2025	36	8	1	45
Jun 2025	57	10	3	70
Jul 2025	29	11	4	44

Viewed (geographical distribution)

Total article views: 883 (including HTML, PDF, and XML) Thereof 883 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 15 Jul 2025

Short summary

Atmospheric variables from climate models often present biases relative to the past. In order to use these models to assess the impact of climate change on processes of interest, it is necessary to correct these biases. We tested several Multivariate Bias Correction Methods (MBCMs) for 5 physical variables that are input variables for 4 process models. We provide recommendations regarding the use of MBCMs when multivariate and time dependent processes are involved.


Total:	0
HTML:	0
PDF:	0
XML:	0