On the skill of high frequency precipitation analyses

Introduction Conclusions References Tables Figures


Introduction
Reliable precipitation analyses and forecasts with both high temporal update frequency and high spatial resolution are essential for many applications.For example, hydrological models usually require gridded precipitation fields on small scales and short lead times which form the major component of flood warning systems (Komma et al., 2007).
In climate research, precipitation re-analyses performed over decades are employed to estimate return periods or other extreme value statistics and are therefore of high social and economic relevance.Gridded precipitation analyses are also gaining importance in the field of spatial verification of numerical weather prediction (NWP) models, especially since convection-resolving models allow for simulating small-scale convective storms.Introduction

Conclusions References
Tables Figures

Back Close
Full A common way of validating the skill of precipitation analyses is the method of leaveone-out cross-validation (LOOCV).However, this method has drawbacks: it is computationally expensive, it assumes a random distribution of the stations with respect to climatology and topography, the results depend on the local conditions of the stations, and -due to the often inhomogeneous and sparse station networks -small-scale features are usually not captured.
Due to its limited representativeness, traditional point-wise verification against station measurements is not adequate and is amended by spatial verification methods like the Structure-Amplitude-Location (SAL) method (Wernli et al., 2008).These novel verification methods require gridded precipitation analyses, preferably model-independent, of high quality as a reference.Wittmann et al. (2010) have used high resolution precipitation analyses to validate the skill of different limited area models (LAM) during a convective season.Similarly, Sattler and Feddersen (2005) have applied daily precipitation analyses to evaluate the quality of a limited area and a global ensemble system during heavy precipitation events.
A variety of methods exists which aim at generating realistic and skillful precipitation analyses.Goudenhoofdt and Delobbe (2009) have shown that the combination of both radar derived precipitation estimates and rain gauge measurements is superior to the individual fields because particular strengths are emphasized, and weaknesses are compensated.Although it is unquestionable that generally, such combination methods improve the skill of quantitative precipitation analysis, their results strongly depend on the local environment (e.g.orography), the quality of the radar and rain gauge data, the scale of interest (e.g. for catchment size scales) and the respective application of the precipitation analysis (Rossa et al., 2005).Thus, the impact on validation results of NWP models can be large and should be taken into account (Rezacova and Sokol, 2002) depending on e.g. the radar-rain gauge combination scheme and the diverse application fields.An overview of radar-rain gauge merging algorithms has been elaborated within the COST 717 project (Rossa et al., 2005), some of them employ bias adjustments schemes (Pereira et al., 1998;Chumchean et al., 2006;Overeem et al., Introduction Conclusions References Tables Figures

Back Close
Full 2009), Kriging approaches (Krajewski, 1987;Sun et al., 2000) also including Bayesian techniques (Handcock and Stein, 1993) and regression-type algorithms (Gregow et al., 2013).A few merging algorithms are of multi-source nature, including radar and rain gauge data and additional components like NWP data to improve the analysis skill (e.g.NIMROD system by Golding, 1998;INCA system by Haiden et al., 2011) Kirchengast et al., 2014).The WegenerNet consists of 151 stations, installed on an almost regular grid in the south-eastern part of Austria (Styria).The topography varies in altitude between 250 and 500 m (Fig. 1).The WegenerNet dataset has already been successfully applied to validate temperature, humidity, and wind speed analyses in an operational context (Kann et al., 2011).Furthermore, the dense station network allows for a thorough evaluation of INCA precipitation for small-scale, convective precipitation patterns.
Section 2 introduces the rapid-INCA analysis module and the station network We-generNet.Section 3 briefly illustrates the synoptic conditions of selected cases with heavy precipitation in August and September 2011, and their skill scores.Section 4 describes the results of a long-term validation during the whole convective period from April to September 2011, followed by a conclusion.Introduction

Conclusions References
Tables Figures

Back Close
Full 2 Data and methods

The rapid-INCA precipitation analysis
The rapid The rapid-INCA precipitation algorithm merges rain gauge measurements from approximately 270 TAWES stations with radar derived precipitation estimates.The synthesis is designed to combine the strengths of both data sources, i.e. the quantitative accuracy of the station measurements and the detailed spatial information of the radar image.However, the algorithmic synthesis has also to cope with the weaknesses and error sources of both measurement methods and -as far as possible -to compensate for them.These weaknesses are predominantly the potential low representativeness of site-specific measurements and the general quantitative uncertainty of precipitation estimates from radar reflectivity.
The precipitation analysis consists of the following steps (cf.comparison with radar data and with neighboring stations, etc. Figure 2 shows the operational rapid-INCA domain and the distribution of automatic stations as well as the position of the five radar locations. 2. Radar derived quantitative precipitation estimates (QPE): ZAMG receives twodimensional radar data from the Austrian aviation service (Austro Control) from five locations in Austria (Patscherkofel, Zirbitzkogel, Schwechat, Feldkirchen, Valluga) with a temporal resolution of five minutes.QPE, derived by the "maximum constant altitude plan position indicator" (max-CAPPI) approach, are bi-linearly interpolated onto the INCA grid.A pre-scaling of the radar data is conducted before a high quality analysis can be calculated as precipitation estimates of the radar may underlie important systematic errors (amongst others due to topographic effects).The local scaling factor results from the ratio of monthly precipitation sums of station interpolation to monthly precipitation sums of radar derived QPE.To avoid unrealistically high scaling factors a maximum value of 2 is set.In addition to the fixed scaling, a latest-data scaling procedure is applied using recent radar and observation data.
3. Combination of weather station interpolation and re-scaled radar field: the combined field is generated by a weighted relation between both fields and leads to a superior precipitation distribution in space than each individual field.It is assured that the observed measurement at the station location is reproduced (within the resolution limits).The larger the distance to the stations, the higher are the weights of the (scaled) radar field.On the other hand, lower radar data quality due to topographic shielding gives higher weight to the interpolated station data.Additionally, elevation effects are parameterized accounting for the increase of precipitation amounts with height (Haiden and Pistotnik, 2009).interpolation.The radar derived QPE contributes with small-scale convective cells which were not captured by TAWES stations of ZAMG.

The WegenerNet
The following brief description of the station network WegenerNet, operated by the Wegener Center for Climate and Global Change of the University of Graz, Austria, is based on Kirchengast et al. (2014) and Kabas (2012), wherein detailed further information can be found.The WegenerNet comprises 151 meteorological stations within an area of about 20 km×15 km in southeastern Styria, Austria (centered near the city of Feldbach, 46.93 • N, 15.90 • E), which is a region with high weather variability and sensitivity to climate change (Kabas et al., 2011a).The stations are arranged on a quasiregular 1.4 km × 1.4 km grid (Fig. 1) and measure the main parameters air temperature, relative humidity, and precipitation amount.Selected stations additionally provide measurements of wind and soil parameters.gridded data (5 min data) are also averaged (summed up for precipitation) to various weather and climate data products (from half-hourly up to annual).
For application purposes the resulting data and further information on the station network are available for users at the WegenerNet data portal (www.wegenernet.org) in near-real time (data latency less than 30 to 90 min).The WegenerNet provides highly resolved individual station data and regular grids since 1 January 2007 as a new data source for research projects investigating local-scale weather and climate and environmental change.Moreover, the data records serve as information source for various applications in the study region (Kirchengast et al., 2014;Kabas et al., 2011b).

Method
The WegenerNet measurements (precipitation data at 5 min resolution) serve as reference in the present study and thus most of the standard comparison techniques are carried out at these station locations.The INCA related fields, rapid-INCA analyses, radar derived QPE and rain gauge measurements (TAWES), are interpolated bi-linearly from the 1 km × 1 km INCA grid to the WegenerNet station locations.This interpolation method is reasonable, as the WegenerNet grid is nearly regular with a spatial resolution comparable to the rapid-INCA grid resolution.
Besides bias, mean absolute error (MAE) and root mean squared error (RMSE), the skill scores Equitable Thread Score (ETS), True Skill Score (TSS) and Frequency Bias Index (FBI), which are commonly used for validating precipitation, have been computed (Table 1).
For spatial comparisons, inverse distance weighted (IDW) interpolation has been applied to obtain WegenerNet measurements on the INCA grid.A quadratic distance weighting function has been chosen by taking into account the five nearest neighbors as the station density of WegenerNet in the target region is relatively high and the respective observations should not be smoothed too much.The resulting field has been processed to obtain the spatial verification indicators structure, amplitude and location

Conclusions References
Tables Figures

Back Close
Full (SAL; Wernli et al., 2008), but also to demonstrate the spatial patterns of standard verification measures.

Heavy precipitation case studies
For the selection of cases with heavy precipitation during the convective season of 2011 the definition of Wussow (1922) for integration times smaller than 30 min has been followed: where t is the time in minutes and h n the amount of precipitation.Thus, a heavy precipitation event is characterized by precipitation amounts exceeding 5 mm in 5 min, 7 mm in 10 min or 12 mm in 30 min.

Synoptic situations
Four cases with heavy precipitation over the WegenerNet region were selected.Figure 4 shows 2 h sums of the 5 min rapid-INCA analyses of these selected cases.
-3 August 2011, 21:00-23:00 UTC Austria was affected by an upper-level flow from the southwest, thus putting it within a warm air mass.A baroclinic zone was gradually approaching from the west, leading to thunderstorms predominantly in the western alpine regions of Austria but also in the south-eastern parts of the country in the late evening.
-15 August 2011, 15:00-17:00 UTC A cold front, associated with a low pressure system centred over southern Sweden, crossed the country during the day, producing considerably lower than normal temperatures and significant rain.Introduction

Conclusions References
Tables Figures

Back Close
Full -19 August 2011, 13:00-15:00 UTC Thunderstorms were widespread due to a cold front crossing the country.A baroclinic zone led to instability with thunderstorms developing, especially in the south of the country.
-1 September 2011, 16:00-18:00 UTC Austria was located in a warm, moderately moist westerly flow.The atmospheric instability, especially in the alpine region and in the south, lead to the development of thunderstorms.

Time series of rapid-INCA analyses and WegenerNet data
The precipitation rates (per five minutes) of WegenerNet measurements, rapid-INCA, radar derived QPE, and TAWES station measurements have been averaged over the WegenerNet domain to show the temporal evolution of the precipitation cells in the four selected cases (Fig. 6).Within the WegenerNet area, only two TAWES stations are located (see Fig. September 2011).The latter situation is triggered by a slight overestimation of radar derived QPE.While this simple validation approach is not suitable for a detailed quantitative analysis, it gives a qualitative view of the four cases under investigation.

Verification measures of rapid-INCA for the selected cases
The verification measures relative bias, MAE, and RMSE (scaled by the mean observed precipitation at each of the WegenerNet stations) have been computed and averaged over space to evaluate the error characteristics of rapid-INCA.Only time steps with a minimum observed precipitation of 0.1 mm/5 min have been selected for the computations.September 2011 at 16:45 UTC, and the interpolation results in an exaggeration of the precipitation field (compared to the rapid-INCA analysis in Fig. 5).An IDW interpolation with an exponent higher than 2 (instead of 1/r 2 ), would limit the spatial influence of the TAWES stations and improve the results in regimes with local convection.Averaged skill scores, FBI, TSS, and ETS, (for a threshold of 0.5 mm/5 min at WegenerNet stations) are shown in Fig. 9.In the case of low representativeness of the TAWES station interpolation (1 September 2011), the scores FBI and ETS from radar derived QPE yield better values than those of rapid-INCA.Hence, the station contribution is decreasing the skill.However, in the majority of cases, the skill of rapid-INCA is higher than of pure radar derived QPE.Higher thresholds than 0.5 mm/5 min lead to worse results of the scores which can partly be explained by the decreasing sample size.Another reason might be the tendency to miss heavy precipitation events with rapid-INCA.
For an objective analysis of the four cases, we applied the Structure-Amplitude-Location (SAL) method (Wernli et al., 2008) to each time step within the respective 2 h intervals.The results are averaged and plotted in Fig. 10  (e.g.IDW with a higher exponent, or more sophisticated interpolation methods such as Kriging by using radar data fingerprints).The location indicator of Fig. 10 does not yield conclusive results as it is relatively low for each of the rapid-INCA constituents.This behavior may be explained by the small area under investigation and thus limited errors in displacement of cells.

Long-term validation results using WegenerNet data as reference
For the long-term validation, rapid-INCA analyses, radar derived QPE, and TAWES station interpolation from 1 April 2011 to 30 September 2011 at 5 min time steps have been interpolated to the WegenerNet stations (see Sect. 2.3).The relative bias, MAE, and RMSE (scaled by mean measured WegenerNet precipitation) have been computed for each of the WegenerNet stations and, for better spatial representation, interpolated to the INCA domain (by IDW).
Only time steps with measured WegenerNet precipitation exceeding a certain threshold have been used to avoid falsifying the error measures with precipitation-free time steps.Figure 11 presents the results for a selected threshold of 0.5 mm/5 min.
The bias shows substantial underestimation of the radar derived QPE, with no specific spatial variation.In contrast, interpolated rain gauge measurements exhibit a better agreement to observations in the vicinity of the two TAWES stations than elsewhere.The rapid-INCA field also shows an underestimation of precipitation higher than 0.5 mm/5 min but with better results near the stations.Especially the TAWES station of Feldbach (further north) has a positive impact on the bias of rapid-INCA.Note that the larger positive bias at one station in the northern part of the area is due to erroneous measurements of the WegenerNet stations.
MAE and RMSE are similar error measures and also show similar characteristics in Fig. 11.With RMSE emphasizing large errors, the spatial distribution of the errors is more pronounced.Again, no significant spatial variation can be indicated for radar derived QPE, whereas interpolated TAWES station data and rapid-INCA exhibit a better Introduction

Conclusions References
Tables Figures

Back Close
Full performance around the Feldbach station.The TAWES station further south (Bad Gleichenberg) yields worse results which may be attributed to the topography in this region: Bad Gleichenberg is surrounded by hills to the north, east, and west (Fig. 1).
Figure 12 shows the skill score results for a threshold of 0.5 mm/5 min.Clearly, there is a tendency to underestimate the precipitation amounts (FBI < 1) for all components; the best results for FBI are obtained close to the TAWES station of Feldbach.TSS indicates more hits than misses (TSS closer to 1) near the stations.ETS yields best results for the rapid-INCA analysis.To investigate the influence of the threshold on the error measures and skill scores, mean values of the error measures and skill scores have been calculated for several thresholds (Fig. 13).
The bias of rapid-INCA increases for increasing thresholds of selected data.Thus, there is a pronounced underestimation of heavy precipitation events.Interpolated rain gauge measurements yield a lower negative bias compared to rapid-INCA which can be attributed to the relatively better performance near the stations, whereas rapid-INCA shows a spatially more homogenous distribution of the bias (compare Fig. 11).Generally, the variation in error measures and skill scores for the TAWES station data is much higher than for rapid-INCA analysis and radar derived QPE.Averaging over the WegenerNet domain can lead to better performance of the error measures and skill scores.
Note that the MAE increases with the threshold whereas the RMSE decreases.This behavior indicates that large errors mostly occur for samples including light precipitation amounts (with RMSE putting higher weight on outliers).For thresholds of up to 1 mm (FBI), 0.5 mm (TSS) and 0.2 mm (ETS) the skill scores show best results for rapid-INCA.At higher thresholds the TAWES stations exhibit better scores than the combined product.During heavy precipitation events, the interpolated rain gauge measurements usually overestimate the spatial precipitation amount and yield better scores than the radar derived QPE which usually underestimates the precipitation field.Additionally, the merging of radar QPE and TAWES station data consists of non-linear algorithms which cause rapid-INCA to converge to the radar QPE Introduction

Conclusions References
Tables Figures

Back Close
Full for heavy precipitation (due to the non-representative behavior of rain gauge measurements during convective events).

Conclusions
In the present study, the performance of short-duration, high-resolution precipitation analyses has been elaborated by means of a set of convective events and a long-term validation covering the convective season in 2011 (1 April 2011-30 September 2011).
In order to point out the small-scale features of convective events, the dense station network of WegenerNet, which is located in the south-eastern parts of Austria (Styria), has been used as a reference.The WegenerNet comprises 151 stations on a grid of about 1.4 km × 1.4 km, and thus operates on a similar resolution as the rapid-INCA analysis system (1 km × 1 km).
The validation results show a general underestimation of rapid-INCA and its constituents (radar derived QPE and rain gauge measurements of TAWES).The spatial variation in error measures is highest for the interpolated TAWES station data.Results from the four selected cases in August and September 2011 show that the contribution from TAWES station interpolation can either have a positive or negative impact on the rapid-INCA skill, depending on the representativeness of the station measurements.Merging TAWES station data with radar derived QPE is able to reduce this effect, but is not able to avoid it completely.
This study indicates that the station contributions play a crucial part in the perfor- an improved pre-scaling of radar QPE may be useful since radar QPE shows a strong underestimation over the whole dataset.Apart from further improvements by applying more sophisticated radar-rain gauge blending methods, the quantification of the uncertainties related to the representativeness problem is a key issue in the generation of an ensemble of precipitation analyses.
The present study reveals that the WegenerNet, which offers high-quality station measurements on very high temporal and spatial resolution, is ideally suited to further improve precipitation analyses and to assess their skill and uncertainty.Figures Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Haiden et al., 2011 for a detailed description): 1. Interpolation of rain gauge data: the 1 min measurements are aggregated to 5 min sums and interpolated by inverse-distance weighting onto the 1 km INCA grid by using the eight nearest stations.Note that only those measurements are used which fulfill several quality control criteria including time-series control, Discussion Paper | Discussion Paper | Discussion Paper | Figure 3 illustrates the combination algorithm in the case of 25 June 2014, 10:15 UTC.In areas with low radar quality, the combination algorithm assigns large weights to the station Discussion Paper | Discussion Paper | Discussion Paper | Furthermore, air pressure and net radiation are observed at one reference station.The collected data are processed by the automatic WegenerNet Processing System (WPS).The raw data are stored by Internet loggers (GeoPrecision GmbH, Germany; www.geoprecision.com) and transferred via GPRS to the database at the Wegener Center Graz.The GPRS transmission is performed hourly, with subsets of about 30 stations transferring in stacked 5 min batches during the first half of the hour.The incoming data files are stored in a database and are checked by the Quality Control System (QCS).The QCS is run hourly and checks for each of the 151 stations the availability and correctness as well as the technical and physical plausibility of the measured data in eight quality-control (QC) layers.In the present article, only data with QC flag of 0 were used, indicating data with the highest quality.In the Data Product Generator, gridded data of the main parameters are derived on a regular 200 m × 200 m Universal Transverse Mercator (UTM) grid from individual station measurements by Inverse Distance Weighting.Subsequently, station data and Introduction Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper |

Figure 5
Figure5shows the spatio-temporal distribution of 5 min rapid-INCA analyses in the region of the WegenerNet which is indicated by the black rectangle.On 3 August 2011, the maximum precipitation amounts are between 2 and 3 mm per 5 min at 22:05 UTC, and then decrease with time.The precipitation cells on 15 August 2011 are gradually intensify with time to 6 mm per 5 min.On 19 August 2011, a heavy precipitation cell moves slowly across the domain, and on 1 September 2011 extremely high maxima are reached (> 10 mm/5 min) before the precipitation cells leave the WegenerNet domain to the south-east.
1) and contribute to the interpolated stations field.Generally, both the onset and evolution of rapid-INCA precipitation amounts follow the WegenerNet observations.However, rapid-INCA underestimates the average precipitation rate in three of the cases, and shows an earlier onset and overestimation in the last case (1 Discussion Paper | Discussion Paper | Discussion Paper | Figure 7 shows the resulting error measures along with the standard deviation (SD) indicated by the error bars.A negative bias is visible for all rapid-INCA constituents except for the TAWES station interpolation on 15 August 2011.This is in accordance with the findings in Fig. 6.The error measures MAE and RMSE are similar for rapid-INCA and radar derived QPE, indicating that the radar derived QPE errors predominantly contribute to the rapid-INCA analysis errors.In certain cases (e.g. 15 August 2011 and 1 September 2011), the rapid-INCA analysis error is larger than the radar derived QPE error.It indicates that the inclusion of the TAWES station observations may decrease the skill, i.e. the TAWES station observations are not representative for this specific precipitation event.Moreover, the variation in the error measures across the WegenerNet domain is large for the TAWES stations, specifically on 15 August 2011 (only two TAWES stations are located within the WegenerNet area and constitute the interpolated stations field; see Fig. 1).The comparison with the WegenerNet stations results in high variability of bias, MAE, and RMSE on 15 August 2011 and 1 September 2011, which demonstrates the low representativeness of the TAWES station field.In such cases, the rapid-INCA analysis (i.e.combination of station interpolation and radar derived QPE) yields worse error measures than the pure radar derived QPE.The high variability of rain gauge measurements (TAWES) can also be seen in the spatio-temporal distribution as shown in Fig. 8.The two TAWES stations are hit by a heavy (localized) precipitation cell on 1 Discussion Paper | Discussion Paper | Discussion Paper | with error bars indicating the SD of the SAL time series.As already emphasized in previous figures, the amplitude values of rapid-INCA show an underestimation of the observed precipitation in all but one case.Radar derived QPE exhibits a higher underestimation than rapid-INCA which demonstrates the positive effect of merging interpolated rain gauge measurements (TAWES) with data of radar QPE.Only on 1 September 2011 did the TAWES station data significantly overestimate precipitation, and in turn over-compensate the radar derived QPE underestimation to finally yield a positive amplitude value of rapid-INCA.Positive structure values indicate too large and/or too flat precipitation cells.rapid-INCA overestimates the extent of precipitation cells (on average, as does the TAWES station interpolation).Radar derived QPE in contrast yields negative structure values, and thus underestimates the extent of the cells.These results suggest that the interpolation method of rain gauge measurements (TAWES) should take into account the current convective situation to be more confined for cases of heavy precipitation Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | mance of the rapid-INCA analyses or, in general in any radar-gauge merging method.Depending on the prevalent synoptic situation, e.g.local convection or large scale precipitation, it may prove useful to adapt the station interpolation algorithm accordingly.Instead of a static IDW with both a fixed number of included nearest stations and a fixed exponent it could be advantageous to apply an IDW with dynamically adjusted parameters.Thus, further studies are needed to investigate the influence of IDW parameters as well as modifications in the combination algorithm on the validation results.Discussion Paper | Discussion Paper | Discussion Paper |
. The Integrated Nowcasting through Comprehensive Analysis (INCA) system has been developed at the Central Institute for Meteorology and Geodynamics in Vienna, Austria (ZAMG) and is in operational use since spring 2004.Besides precipitation (the most traditional nowcasting parameter) many different parameters are computed by INCA (e.g.precipitation type, temperature, humidity, wind etc.).The techniques for computing analyses and nowcasts vary from parameter to parameter, as well as temporal resolution and update frequency.In the present article, the INCA precipitation analyses are validated against the independent dataset of the WegenerNet climate station network (operated by the Wegener Center for Climate and Global Change, University of Graz, Austria;