We introduce a hierarchical Bayesian model for the spatial distribution of rainfall corresponding to an extreme event of a specified duration that could be used with regional hydrologic models to perform a regional hydrologic risk analysis. An extreme event is defined if any gaging site in the watershed experiences an annual maximum rainfall event and the spatial field of rainfall at all sites corresponding to that occurrence is modeled. Applications to data from New York City demonstrate the effectiveness of the model for providing spatial scenarios that could be used for simulating loadings into the urban drainage system. Insights as to the homogeneity in spatial rainfall and its implications for modeling are provided by considering partial pooling in the hierarchical Bayesian framework.
For an existing urban drainage network, a proper consideration of the spatial structure of extreme rainfall events is important for an assessment of the effectiveness of the network for handling urban flooding subsequent to rainfall events of varying duration, especially as concerns regarding the resilience of the system under a changing climate emerge. Often, investigators focus on return period analysis of extreme rainfall at a site considering annual maxima or peaks over threshold for a specific rainfall duration. In a regional context, spatial models of annual maximum rainfall are sometimes considered (Renard et al., 2006; Renard and Lang, 2007; Dyrrdal et al., 2015). However, since the annual maximum is unlikely to occur for a given event at all sites, these models do not represent the actual structure of potential extreme rainfall events. Thus, existing models for spatial rainfall extremes cannot be used to provide forcing for the performance of an existing drainage network (natural or constructed) under an extreme rainfall event. We address this situation in this note by considering that the rainfall events of interest for a specified duration are ones where any one of the sites in the region experiences an annual maximum event; the spatial field or rainfall of interest is then the field associated with each such event.
In the exploratory analyses performed for New York City, we noted that the
structure of storms that lead to annual maximum events at different gages in
the region may not be the same and that the basic statistics of rainfall
vary across sites. We assemble a dataset of the annual maximum rainfall for
each specified duration at each station. Let us denote this as
In Sect. 2, we present the data and the context for the application to the greater New York area. In Sect. 3, we describe the details of the multivariate hierarchical Bayesian models. The results are discussed in Sect. 4. Finally, in Sect. 5, we present a summary and conclusions.
The greater New York City area has a high density of man-made infrastructure and hence a complex hydrological landscape. Like many older cities, sanitary and industrial wastewater, rainwater, and street runoff are collected in the same sewers and conveyed together to treatment plants. Approximately 60 % of NYC's drainage area is served by these combined sewers. Flooding and combined sewer overflows (CSOs) are a concern, and innovative solutions for using the sewer system itself as flood storage by pumping water to different areas during a storm has been suggested. Such hydrologic system upgrades need to be informed by the spatial variability of extreme precipitation.
There are very few rain gage records that are longer than 20 or 30 years. Persistent data quality issues further reduce the available data. This challenge of reconciling sparse data with spatially variable hydrological networks and meteorological phenomena is common to many urban areas. Widely accepted design standards are derived from a set of intensity–duration–frequency curves developed using daily rainfall records from the period 1903–1951 (The New York City Department of Environmental Protection, 2008). An analysis of precipitation extremes for the region is offered in Wilks and Cember (1993), using daily rainfall data, and in McKay and Wilks (1995), using hourly rainfall data. None of these analyses consider the spatial correlation of rainfall.
The precipitation data was obtained from the National Climatic Data Center (National Climate Data Center, 2013). Rain gages were selected based on their proximity to New York City, data quality, and length of the historical record. Twenty-nine stations were initially identified as lying within a 100-mile radius of Central Park, with over 25 years of continuous hourly rainfall records. Sixteen stations were excluded, since the resulting data quality was too poor. The final dataset consists of the remaining nine stations (Table 1); abbreviations for each station used in the figures throughout are provided in the first column.
Rain gage stations in New York City and surroundings.
Percent of simultaneous or near-simultaneous annual maxima events shown for the site-by-site comparison for nine sites and 1, 6, 12, and 24 h storms.
The Shapiro–Wilks test (Shapiro and Wilk, 1965; Royston, 1992) was applied to the log-transformed annual maxima series at each station for each storm duration from 1 to 24 h. For these 216 time series, the null hypothesis of the appropriateness of the log-normal distribution was not rejected for 98 % of the sites at the 1 % level, 88 % at the 5 % level, and 81 % at the 10 % level. Though other distributions – such as the generalized extreme value (GEV), Pearson, and log-Pearson type III (LP3) – are popular for extreme precipitation modeling, in our application to the spatial field of rainfall, only one of the sites experiences the annual maximum event, and others may not be extreme values. In such a setting, the log-normal distribution can often be an appropriate representation (Raiford et al., 2007), as seen here. Consequently, to illustrate the idea, we consider a log-normal distribution with spatial correlation across the sites for the NYC example. Other choices could very well be made.
A heat map showing the fraction of annual maximums that occur simultaneously
is provided in Fig. 1. For these plots, we define simultaneous storms to
be those beginning within
This diagnostic analysis highlights the importance of considering the spatial structure of extreme rainfall for an event with a specified duration.
A hierarchical Bayesian approach that provides the ability to partially pool model parameters across the rain gage sites was developed. Full pooling would imply that a parameter (e.g., the mean or variance of the distribution) was homogeneous across the sites. No pooling would imply that each site was independent. Partial pooling is an intermediate step that allows information to be shared across sites at a level informed by the data. This results in a multi-level model, where model parameters are estimated at each site but are assumed to be drawn from the parameters of distributions that are specified at the regional level for each parameter (Gelman and Hill, 2007). Such an approach has been implemented for hydrometeorological extremes in Lima and Lall (2010) and in Kwon et al. (2008), and for paleoclimate reconstructions by Devineni et al. (2013).
In this model, we consider a conditional process – where site
There are nine stations, and therefore there are nine distinct datasets
We consider a subset of the previous model where the assumption that the
mean log rainfall drawn from a common spatial mean is relaxed. This leads
to the simpler, no-pooling model represented below:
The spatial fields model can be used to simulate rainfall fields
corresponding to an annual maximum occurring at any one of the sites,
Two hundred and sixteen models (one for each duration and each site) were
fitted using JAGS (Just Another Gibbs Sampler) (Plummer, 2003; Denwood et al.,
2016). It uses a Markov chain Monte Carlo (MCMC) simulation algorithm (a
Gibbs Sampler for the current example) to simulate the posterior
probability distribution of parameters. A random normal distribution was
used for the vector of station means (
Empirical cumulative distribution function of the posterior
distribution of
For each model, the convergence of the posterior distribution of each parameter was checked using the shrink factor proposed by Gelman and Rubin (1992) – values under 1.1 for all parameters suggest that the model has converged. Convergence plots (showing the mixing of the four chains) were visually checked for all cases. All models converged appropriately, with each parameter attaining a shrink factor between 1.0 and 1.1 and with the large majority reaching 1.0.
Empirical cumulative distribution function of the 12 h 10-year return period event from SF hierarchical models for the nine stations (Central Park, LGA, New Milford – top to bottom, left panel; Essex Fells, New Brunswick, Staten Island – top to bottom, middle panel; JFK, Newark, Watchung – top to bottom, right panel). The empirical cumulative distribution function derived from the observed 12 h 10-year return period field is also shown. The band depicts the 99 % credible interval of the posterior simulations.
We compare the performance of the two SF models using the deviance information criterion (DIC) and pD, as recommended in Gelman et al. (2004). The scores were virtually identical for the two types of SF models for each rainfall duration. Next, we considered whether the common mean in the hierarchical model converged as successfully as other parameters; it did. Gelman and Hill (2007) suggest that, when there are only a small number of groups and when the group-level standard deviation is large, multi-level modeling may not add much information. The resulting model will not necessarily perform worse and will likely resemble the model without pooling (as it does here). The posterior parameters for the resulting simulations are essentially identical (Fig. 2a), and the posterior for the mean shrunk only very slightly (Fig. 2b).
Empirical cumulative distribution function of 5-year
return period events for 1 h
Next, we consider how the return period spatial rainfall fields identified in the SF hierarchical models compare to observed return period spatial fields. We do so by plotting the empirical cumulative distribution function of the return period event field estimated from the hierarchical model and by comparing it with the empirical cumulative distribution function of the return period event field derived from the observed data. The 99 % credible interval obtained from the posterior simulations of the hierarchical models is also presented to represent the uncertainty. Though the Bayesian models can easily simulate any return period, the reliability of the empirical estimate is dependent on the length of the record, so it is only reliable as a goodness-of-fit measure for shorter return periods. Results from all nine stations for the 10-year 12 h accumulated rainfall are presented in Fig. 3. The plots for all nine stations at the 1 and 24 h durations and the 10-year return period are provided in Figs. A1 and A2, respectively, of the appendix.
The observed 10-year 12 h return period fields for all stations are within the 99 % credible interval of the simulations, for the most part. The SF model simulated tails that are much greater in magnitude than the observed, as expected. The uncertainty band is also wider at the tail end of the distribution. New Milford is an exception, with slight underprediction of the left tail. These differences are amplified at the left tail for higher durations (i.e., the 24 h storms). The shorter durations (1 h storms) seem to be modeled well, albeit with wider credible intervals.
Figure 4 presents a slightly different view of the results, with a focus on how well various durations for a shorter return period event (5-year events) are simulated for a specific station – Staten Island. While 1 h 5-year return period storm fields are well simulated, there seems to be a bias (with the observed field's left tail falling outside the 99 % credible interval of the simulated) with increasing durations. It is difficult to identify exactly why this might be the case without significant additional exploratory analysis of the Staten Island data. However, we note that the simulations reflect additional information provided by the other stations in the model. In this application, the Staten Island site record is 20 years shorter than the record at the other sites, and the shift in the simulations reflects the shift in the rainfall in the other sites over this period. Thus, the pooling strategy is instrumental in using the joint distribution across sites from the common period of record to provide simulations that cover the shift in the period. Of course, if the correlation across sites has also changed over the period of missing data, the algorithm is incapable of replicating such a change.
For larger cities, a consideration of the drainage network and the spatial dependence in rainfall at different durations is important to consider, at least from the perspective of assessing the performance and resilience of the network and perhaps also for design considerations. We were interested in formulating and testing a simple model that could directly explore whether or not and to what extent there was opportunity to pool regional information on extreme rainfall events to describe plausible spatial fields of extreme rainfall. This led to postulating and testing a Bayesian model that considers the spatial field of rainfall associated with an annual maximum occurrence at any site. We considered the application of the model to relatively long rainfall time series from the New York City region. Initial exploratory analyses suggested that the rainfall characteristics and storm tracks varied by event and by season across the region, such that distinct clusters could be identified, suggesting that the region had a heterogeneous spatial structure with respect to extreme rainfall (Hamidi et al., 2017). Our applications further clarified the nature of this heterogeneity. It is interesting to also note from the New York City analysis that there is support for pooling the spatial covariance of rainfall across all sites (irrespective of which one experienced an annual maximum rain event for a given duration), even though, often, the exceedance probability distributions of rainfall for a given duration may differ across sites, even after partial pooling. The hierarchical Bayesian framework permits a consideration of the uncertainty in parameter and model structure and helps us to identify the level of homogeneity that may be appropriate for representing the processes underlying a particular dataset.
Rain gages are the preferred data source for extreme event modeling because of their long record, but incorporating radar in addition to rain gages could provide the spatial density needed to explore how event rainfall characteristics relate to specific meteorological phenomena or to provide comparable simulations to existing stochastic models. The radar information would contain considerably more spatial detail necessary for building the type of model exemplified here. However, radar rainfall records are much shorter, and consequently, one needs to develop a methodology to appropriately blend the shorter but spatially richer radar data with the longer but spatially sparse gage data. Our algorithm can be readily applied to a mix of radar and rain gage data. However, some extensions need to be pursued to address the very different record lengths of each data source.
We used a log-normal distribution applied to rainfall for each duration to illustrate our approach. The goodness-of-fit tests support this assumption, and this permits some confidence in the kind of conclusions we drew from the applications to the New York City data. However, other models – such as the GEV or generalized Pareto or other choices for the distribution – could very well be considered. The point here was to highlight the need to consider spatial covariance and an appropriate blending of local and regional data sources through partial pooling.
This appendix includes Table A1 and Figs. A1 and A2. Table A1 provides an example derivation of extreme rainfall fields. Figures A1 and A2 provide the results and baseline comparisons of the 1 and 24 h 10-year return period events from the SF models for the nine stations in and around New York City.
Illustration of spatial rainfall fields for an example
year, 1979, for 12 h accumulated rainfall. Anchor station column is the
condition where annual maximum rainfall is identified
(
Empirical cumulative distribution function of the 1 h 10-year return period event from SF hierarchical models for the nine stations (Central Park, LGA, New Milford – top to bottom, left panel; Essex Fells, New Brunswick, Staten Island – top to bottom, middle panel; JFK, Newark, Watchung – top to bottom, right panel). The empirical cumulative distribution function derived from the observed 1 h 10-year return period field is also shown. The band depicts the 99 % credible interval of the posterior simulations.
Empirical cumulative distribution function of the 24 h 10-year return period event from SF hierarchical models for the nine stations (Central Park, LGA, New Milford – top to bottom, left panel; Essex Fells, New Brunswick, Staten Island – top to bottom, middle panel; JFK, Newark, Watchung – top to bottom, right panel). The empirical cumulative distribution function derived from the observed 24 h 10-year return period field is also shown. The band depicts the 99 % credible interval of the posterior simulations.
The code for conducting the analysis presented in this paper can be made available upon request.
Rainfall data for the analysis of the nine sites in and around New York City
can be obtained from the public source provided in the references: National
Climate Data Center,
ND and UL designed the study and wrote and edited the manuscript. BRM and ND conducted the analysis. BRM wrote the original draft of the paper.
The contact author has declared that none of the authors has any competing interests.
Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The authors would like to thank the editor and the reviewers for their valuable feedback which helped improve this technical note.
This research has been supported by the Department of Energy's Office of Science “Early Career Research Program” (grant no. DE-SC0018124, awarded to Naresh Devineni) and the National Science Foundation's “Water Sustainability and Climate” (WSC) program (grant no. 1360446). Partial support was also provided by the American International Group (AIG) within the framework of the “Climate Informed Global Flood Risk Assessment” project.
This paper was edited by Xing Yuan and reviewed by Xun Sun and two anonymous referees.