Online multistep-ahead inundation depth forecasts by recurrent NARX networks

Various types of artificial neural networks (ANNs) have been successfully applied in hydrological fields, but relatively scant on multistep-ahead flood inundation forecasting, which is very difficult to achieve, especially when dealing with forecasts without regular observed data. This study proposes a recurrent configuration of nonlinear autoregressive with exogenous inputs (NARX) network, called RNARX, to forecast multistep-ahead inundation depths in an inundation area. The proposed R-NARX is constructed based on the recurrent neural network (RNN), which is commonly used for modeling nonlinear dynamical systems. The models were trained and tested based on a large number of inundation data generated by a well validated two-dimensional simulation model at thirteen inundation-prone sites in Yilan County, Taiwan. We demonstrate that the R-NARX model can effectively inhibit error growth and accumulation when being applied to online multistep-ahead inundation forecasts over a long lasting forecast period. For comparison, a feedforward time-delay and an online feedback configuration of NARX networks (T-NARX and O-NARX) were performed. The results show that (1) T-NARX networks cannot make online forecasts due to unavailable inputs in the constructed networks even though they provide the best performances for reference only; and (2) R-NARX networks consistently outperform O-NARX networks and can be adequately applied to online multistep-ahead forecasts of inundation depths in the study area during typhoon events.


Introduction
The increasing frequency and severity of floods caused by climate change and/or land overuse has been continuously reported both nationally and globally, especially in Southeast Asia, in the last two decades.Taiwan is located in the northwestern Pacific Ocean, where the activities of subtropical jet streams are frequent.Typhoon Morakot struck southern Taiwan with a sudden rainfall (the highest rainfall reaching 1166 mm day −1 ) on 8 August 2009, and the extreme rainfall induced vast mudslides and disastrous flooding throughout southern Taiwan.A mudslide buried the whole Xiaolin Village, which caused an estimated 500 deaths.In brief, Typhoon Morakot resulted in 665 deaths, 34 missing, numerous refugees, and roughly NT$110 billion (US$3.3 billion) in damages.In 2010, Typhoons Fanapi and Megi hit southern and eastern Taiwan in mid-September and mid-October, respectively.Both typhoons resulted in loss of life and property, and severely damaged city functions.Flood depth forecasting is an important component of the contingency plan for alleviating flood risk and loss of life and property.
Flooding in urban areas poses a great challenge to hydrologists because of the complex interactions and disruptions associated with nonriverine urban flooding.In the past few decades, simulations of flood inundation extent have been made by the advances in numerical modeling techniques (Bates et al., 1995;Lane, 1998;Marks and Bates, 2000;Bates and De Roo, 2000;Hsu et al., 2000;Wheater, 2002;Kang, 2009) and the use of SAR (synthetic aperture radar) (Bates et al., 2006;Mason et al., 2007;Zwenzner and Voigt, 2009).Conventional inundation models could provide regional hydrogeologic characteristics in response to various patterns of storm events (Hsu et al., 2000;Bates et al., 2003), which are useful information for flood management in early and/or planning stages, nevertheless these models commonly require substantial computational time for iterative solutions to simulate high-resolution spatial flood depths.Consequently, online inundation forecasts could not be effectively conducted by conventional inundation models.The Published by Copernicus Publications on behalf of the European Geosciences Union.

936
H.-Y. Shen and L.-C.Chang: Online multistep-ahead inundation depth forecasts great potentiality of artificial neural networks in hydrological time series forecasting and their encouraging results obtained in literature were many (i.e., Maier and Dandy, 2000;Brath et al., 2002;Chau et al., 2005;Lin et al., 2006;Toth and Brath, 2007;Chen and Chang, 2009;Wu et al., 2009;Abrahart et al., 2012).The majority of the applications are river flow forecasting, nevertheless, there are relatively few researches on online flood inundation applications.Valeriano et al. (2009) stated that inundation areas were estimated using topographic characteristics based on the simulated overflow volumes recorded at the control point downstream.Chang et al. (2010)  Because of extremely limited response time to flood disasters in urban areas of Taiwan, reliable multistep-ahead inundation depth forecasts would be helpful in managing contingencies and emergencies and in alleviating flood risk and loss of life and property.However, online multistep-ahead flood depth forecasts face two challenges.The first challenge involves extending one-step-ahead forecasting to multistepahead forecasting.In one-step-ahead forecasting tasks, ANN models estimate the next sample value without feeding back its output to the model's input layer.In other words, the input contains only observed values.For multistep-ahead forecasting tasks, current inputs would be repeatedly mapped onto various multistep-ahead outputs, or model outputs would be sequentially fed back to the input layer to provide one-step further forecasts.In the latter case, the input layer might contain not only observed values but also model outputs.As known, feeding model outputs back to the input layer makes the model become a dynamic modeling task, which is substantially more complex than static modeling tasks.For these complex tasks, recurrent neural networks (RNNs) play an important role (Menezes Jr. and Barreto, 2008;Chang et al., 2012).RNNs usually incorporate with the architecture of a multilayer perceptron (MLP) for an exploitation of its nonlinear mapping capability (Haykin, 2009).For instance, the simple recurrent network (Elman, 1990) has the outputs of the hidden layer fed back to the input layer, and demonstrates its great ability in extracting dynamic time variation characteristics.In recent years, RNNs have been applied to the field of hydrological modeling (Chang et al., 2002, Chang et al., 2004;Coulibaly and Baldwin, 2005;Besaw et al., 2010;Chiang et al., 2010).
The other challenge is the lack of real-time observed inundation depths, and thus models usually require proceeding with online forecasts through the whole period of a typhoon event (commonly over 20 h in our cases) without any observed flood depths.In the multistep-ahead inundation forecast problem, the error accumulation is a vital reason restraining the accuracy of long-term inundation depth forecasts.As known, a small prediction error at the beginning can accumulate and propagate in the future (Parlos et al., 2000), which results in poor prediction accuracy when models keep on making forecasts for a long time.To solve the lack of observed depths and mitigate error propagation in the long-run, this study proposes a recurrent configuration for a nonlinear autoregressive with exogenous inputs (NARX) network, called R-NARX, to construct multistep-ahead inundation depth forecast models.To verify its practicability, the Yilan County in northern Taiwan is used as the study area and another two types (i.e., time-delay and online configurations) of NARX networks are also implemented for fully exploring their capabilities in multistep-ahead flood inundation forecasts.

NARX network
NARX is an important and useful mathematical model of discrete-time nonlinear systems.Nonlinear systems can be approximated by an MLP network, called NARX, a powerful dynamic model for time series prediction (Jiang and Song, 2011;Menezes Jr. and Barreto, 2008).The architecture of a NARX network based on a multilayer perceptron neural network consists of p antecedent values of exogenous input vectors X(t), such as online rainfall intensity; q antecedent actual values z(t + n − q), such as inundation depths, which are tapped-delay inputs or fed back from the model's output; and a single n-step-ahead output ẑ(t + n).For many practical applications in hydrological systems, such as flood inundation depth forecasting in this study, we commonly face the problem of lacking online data.ANNs are well known datadriven models.In case of no online data available, we have to find a way to train and test the networks step-by-step in real time through the whole flooding period.Bearing this in mind as a motivation, we propose three configurations of the NARX network to fully explore and examine their capabilities of learning and generalization in flood inundation depth forecast tasks.

Learning algorithms
Three types of configured NARX networks are: In the T-NARX network (Fig. 1), all the inputs (p antecedent rainfalls and q antecedent inundation depths) are observed values in both training and testing phases.The T-NARX network is indeed a feedforward time-delay neural network (TDNN), which is a static neural network and frequently used to predict theoretical time series with long-range dependence present in data.The proper synaptic weights of the network can be obtained using the batch mode of the conjugate gradient back-propagation learning algorithm for searching minimum errors during the training phase.The constructed network and its synaptic weights would be fixed in the testing phase.In reality, only real-time rainfall values could be obtained, while real-time observed inundation depths could not be obtained online.Moreover, when inundation depth forecasts are conducted for more than two-hour-ahead (n>1), the q antecedent actual values (z(t +n−1), z(t +n−2),. . ., z(t +n−q)) are future data and cannot be obtained at present time.Consequently, the constructed T-NARX could not conduct online forecasting.The T-NARX network is implemented mainly to find the optimal solution, where the long-range dependences inside the inputoutput patterns could be extracted and the solution could be provided as a reference.
It is interesting to learn the reliability (training data set) and generalization (testing data set) of the models constructed above.Alternatively, we propose the O-NARX to investigate the capability of T-NARX networks constructed in the case of no online q antecedent actual inundation depths available.After the networks have been constructed, the model outputs of inundation depths (ẑ(t)) are fed back to the input layer for online forecasting in both training and testing phases (Fig. 2).It is noted that the O-NARX networks are exactly the same as the T-NARX networks except for two major differences: the q antecedent actual inundations used as inputs in both training and testing phases of the T-NARX models; and the model outputs fed back to the input layer in testing phase of the O-NARX models.
We intended to further solve the problem of no actual inundation depths during the online process and propose to use the model outputs of inundation depths as the inputs in both training and testing phases.In this way, the NARX networks would be trained with imperfect information as well as remaining similar characteristics of input-output patterns in the testing phase, and therefore we argued the configured NARX networks would maintain similar capability of online multistep-ahead forecasts in both training and testing phases.Figure 3 shows that there are p antecedent rainfall and q previous model outputs in both training and testing phases.The observed inundation depth is only used in the training phase to calculate the error and adjust the weights.The constructed network with p antecedent rainfalls and q sequence model outputs as the inputs can be used for online multistep-ahead forecasts.The synaptic weights of the network can be adjusted using the online back-propagation learning algorithm to search for minimum errors on an example-by-example basis during the training phase and eventually reach a stable condition, where the error could not be further deduced and all the synaptic weights then remain the same in the following search process.The constructed network and its synaptic weights would be fixed in the testing phase to evaluate its applicability and reliability in new events (input-output patterns).We notice that the input information based on model outputs is not perfect (real), which could include different fault levels.It is interesting to learn how well the configured networks can perform by learning imperfect inputs.That is, to examine the effect of inhibition ability of the constructed networks on error growth and accumulation.

Mathematical formulation
This study minimizes the energy function by the steepest descent method, and the recurrent configuration is used to adjust the synaptic weights on an example-by-example basis.The mathematical formulation of the R-NARX is presented as follows.
The network contains M exogenous inputs and a single output.For simplicity, assume that p = 0, q = 1, and n = 1 in Fig. 3. Let x(t) denote the M × 1 input vector, ẑ(t + 1) denote the corresponding single output, and y(t + 1) denote the corresponding N × 1 output vector in the hidden layer.The input x(t) and one-step delayed output ẑ(t) are concatenated to form the (M + 1) × 1 vector µ(t), of which the i-th element is denoted by µ i (t).Let A denote the set of indices i, for which x i (t) is an external input, and B denote the set of indices i, for which ẑ(t) is the output of the network.We thus have Let W denote the N ×(M +1) weight matrix of the hidden layer.Let V denote the N × 1 weight matrix of the output layer.The net activity of neuron j at time t is computed by (2) The output of neuron j is obtained by passing net j (t + 1) through the nonlinearity f (.), yielding y j (t + 1) = f (net j (t + 1)). (3) The net output in the output layer at time t is computed by net(t + 1) = v j y j (t + 1), ( 4) Let z(t + 1) denote the target value at time t + 1.The error e(t + 1) is given as, e(t + 1) = z(t + 1) − ẑ(t + 1).( 6) Define the instantaneous value of the network error at time t + 1 as the energy function.
For the sequential model of the back-propagation learning algorithm, the negative gradient (−∇E) of the energy function is used to adjust the synaptic weights at each time step.The weight change for any particular weight v j can thus be written as where η 1 is the learning-rate parameter.Consequently, = −e(t + 1)f (net(t + 1)) ∂ ẑ(t + 1) ∂v j = f (net(t + 1)) The same method is also implemented for weight w j i , where , and η 2 is the learning-rate parameter.The partial derivative ∂E(t+1) ∂w j i can be obtained by the chain rule for differentiation, shown as follows ∂y j (t + 1) As compared with the learning algorithm of TDNN, we would like to note that the proposed algorithm has an additional term in both Eqs. ( 11) and ( 14), in which ẑ(t) is a function of w j i (or v j ), while in TDNN z(t) is not a function of w j i (or v j ) but an observation value.

Study area and materials
Yilan County is selected as the case study.It is the most beautiful urban county near Taipei City, but it has a long history of flooding problems, which continue threatening the lives and livelihoods of its residents.Yilan County, located in northeastern Taiwan, spans an area of approximately 2143 km 2 and is divided by three river basins (Toucheng, Lanyoung and Nan'ao).In the last two decades, Yilan has frequently suffered from flood disasters that resulted in grave losses of agricultural crops and private property.Thirteen villages have been identified as inundation-prone sites by the government project on the renovation of inundation-prone areas.
The thirteen inundation-prone sites are used to investigate the robustness and stability of the proposed models in this study.
Figure 4 shows the thirteen inundation-prone sites and three nearby rain gauges.
Because historical observed data of inundation depths are rare and no historical hydrograph of inundation depths are available for storm events, we have to find other data sets to build forecast models.Fortunately, the synthetic hydrographs of flood depths were obtained from the Water Resources Agency (WRA), Taiwan, which were well validated by the urban inundation model linkage of the HEC-1 model (developed by the Hydrologic Engineering Center for flood hydrograph computations, HEC-1), SWMM (the storm water management model) and the two-dimensional noninertial overland flow simulation model.The urban inundation model proposed by Hsu et al. (2000) was implemented to simulate inundation in urban areas, where the movement of water in the studied area is characterized by the sewer flow component and the surcharge-induced inundation component.SWMM is employed to provide the surcharged flow hydrographs for surface runoff exceeding the capacity of the storm sewers, and the 2-D diffusive overland-flow model considering the non-inertia equation with alternative direction explicit numerical scheme was used to calculate the detailed inundation zones and depths due to the surcharged water on overland surface.The detail description of the urban inundation models (including SWMM and 2-D diffusive overland-flow model) and their application for a 100-yr (return period) 24-h design rainfall of analyzing surface inundation in the Taipei city can be found in Hsu et al. (2000).
There were 24 design rainfall events and 31 historical rainfall events.The 24 design rainfall events are comprised by various return periods of three nearby rain gauges.The 24h design hyetograph pattern of all rain gauges and the cumulative rainfall for various return periods (10-, 25-, 100and 200-yr) in three nearby rain gauges are shown in Fig. 5 and Table 1, respectively.The 31 historical rainfall events are shown in Table 2.The corresponding 24-h flood inundation depth hydrographs to those events (24 design events and 31 historical events) were obtained from the WRA, Taiwan, and used to configure the models.The proposed models of thirteen sites in this study area are then thoroughly trained and tested based on flood inundation data generated by the well validated inundation model with the design rainfall patterns and/or historical rainfall events.The maximums, means and variances of flood inundation depths at 13 sites are shown in Table 3.It appears that the maximums, means and variances of the training case are much larger than those of the testing case at all sites.This is mainly because the return periods of 24 design storms (only used in the training case) are

Results
This study investigates the multistep-ahead inundation forecast performances of three types of NARX networks based on a large number of rainfall-inundation patterns for all thirteen inundation-prone sites in Yilan County.The performances of these three models are evaluated by the root mean square error (RMSE) and the average normalized peak error (ANPE), as shown below. Hydrol where Ĥi and H i are the estimated flood depth and the simulated flood depth of the i-th data, respectively.Ĥ p j k and H p j k are the estimated peak flood depth and the simulated peak flood depth of the j -th event at the k-th inundation-prone site.
For each inundation-prone site, one-to six-hour-ahead (n = 1-6) flood depth forecast models are constructed by the T-, O-and R-NARX networks, and there are 624 training data (26 events ×24 h) and 696 testing data (29 events ×24 h).An inland flood inundation occurs when intense precipitation falls over a short period or precipitation accumulates over several hours.Generally, real-time rainfall information could be obtained, while the observed inundation depth could not be obtained.Consequently, the rainfall is the sole information to evaluate the change of flood inundation and it acts as a driven force to the system as well as to the running model.If rainfall was not included in the forecast models, the model cannot sequentially update the forecast values by itself without the driven force.Therefore rainfall plays a crucial role in the inundation forecast models.Moreover, the antecedent inundation depths represent the influence of the precipitation accumulation on the current inundation depths and the current rainfall causes future inundation depths.The inputs include current and two previous hourly rainfalls (t, t − 1 and t − 2, i.e., p = 2 in this study) of three nearby rain gauges and one hour before the nth-hour forecasted flood depth (t + n − 1, i.e., q = 1 in this study), and the output is the next n step (t + n) flood depth.The input dimension is 10 and the output dimension is only 1.After implementing an intensive trial-and-error procedure based on the training data set, the networks are constructed to have only one hidden layer with three nodes, which in general would have the most suitable performances for all models of thirteen inundation-prone sites.The networks are then applied to the testing data set without further modifications.Moreover, for investigating and emphasizing the influence of training strategy of the R-NARX models on mitigating the error growth, the constructed O-NARX networks recomputed their online forecasts by using the model outputs instead of the actual inundation depths as inputs in the training data.The summarized results of the three kinds of one-to six-step-ahead forecast models are presented in Table 4.It represents the results of one-to six-hour-ahead forecasting for the maximum, minimum, average RMSEs and ANPEs of 13 sites in Yilan County.The maximum (minimum) indicates the maximum (minimum) value occurred at one of the 13 sites, while the mean indicates the average RMSE value of 13 sites.The results show that the RMSEs and ANPEs of T-NARX networks are relatively smaller than the other two networks in all cases.These values indicate the optimal results we might reach based on the perfect anticipant conditions (rainfall and inundation depths).In addition, the shapes of the inundation depth hydrographs generated by three models are similar and their mean absolute We have examined the results of T-NARX and O-NARX, which have exactly the same structures and synaptic weights at each site.T-NARX models have the great performances in all cases, while O-NARX models cannot effectively provide accurate long-term inundation depth forecasts, especially in the testing cases.These results provide clear and rigid evidences that the reliability and generalization of the constructed networks are poor in the training data set and bad in the testing data set if online antecedent inundation depths cannot be obtained, which indeed is the common situation we have.The R-NARX model also has substantially smaller error accumulation and propagation than the O-NARX.
The main reason for the poor performance of O-NARX models is that the inputs of inundation depths of T-NARX and R-NARX models keep consistent in both training and testing phases, while those of O-NARX are different in the training process (actual inundation depths) from the online forecasting process (sequential model outputs in training and testing data).We realize that the ANN has a great capability of learning from input-output patterns and powerful ability of generalization.Our analysis results provide extra evidence that the consistent input-output patterns in both training and testing phases is an important factor which can significantly influence the network capabilities of learning and generalization.
As we compared the results of O-NARX and R-NARX, the RMSEs and ANPEs of the O-NARX are much larger than that of the R-NARX in all cases.For instance, the mean RMSEs of six-hour-ahead forecasts in the testing cases are 0.36 m and 0.24 m for O-NARX and R-NARX models, respectively; and the maximum RMSEs of six-hour-ahead forecasts in the training cases are 0.34 m and 0.23 m for O-NARX and R-NARX models, respectively.Similarly, the ANPEs of six-hour-ahead forecasts in the testing cases are 48 % and 65 % for O-NARX and R-NARX models, respectively.These results indicate that R-NARX networks provide much better (accurate and reliable) forecasts than O-NARX networks.For one-to six-hour-ahead flood depth forecasts, the RMSE growth trends of T-NARX, R-NARX and O-NARX models increase slightly, moderately and dramatically, respectively, in both training and testing phases.
To illustrate the forecasting accuracy of O-NARX and R-NARX models, the scatter plots of the simulated versus forecasted flood depths of site 2 (one-, three-and six-hour-ahead) in the testing data sets are shown in Fig. 6, respectively.In the R-NARX model, almost all pairs of forecasted and simulated points scatter closely to the diagonal line for flood depths in Fig. 6.In the O-NARX model, only one-and three-hourahead flood depths scatter suitably around the diagonal line.According to the results, the R-NARX provides a significantly superior performance to the O-NARX.Figure 7 shows the growth trends of the mean RMSE at 13 sites for one-to six-hour-ahead forecasts of R-NARX and O-NARX models through online forecasts proceeding from the 1st hour to the 24th hour in the testing phase.It is noticed that the model outputs will gradually part from true values as the forecasting proceeds, and feeding those imperfect outputs back to the input layer would further accelerate the growth of forecast errors.R-NARX models can provide accurate long term inundation forecasts and last a long forecasting period with the consistent imperfect inputs in the training process and online forecasting process.The mean RMSE growth trends of R-NARX models increase gradually, whereas those of O-NARX models increase rapidly.This demonstrates that the R-NARX model has substantially smaller error accumulation and propagation than the O-NARX model, and the proposed R-NARX networks can provide reasonable and robust results for multistep-ahead flood depth forecasts if real-time observed rainfall and the feedback of model inundation outputs to the input layer can be implemented as the forecasting proceeds.The results demonstrate that the proposed R-NARX model has the ability to tolerate imperfect inputs and effectively inhibit error growth and accumulation when being applied to multistep-ahead forecasts, and it has an outstanding capability for flood inundation depth forecasts and can proceed online for a long period solely based on real-time rainfall information.The better results obtained by the R-NARX model over the O-NARX model provides sound evidence that the reliability and generalization of the constructed networks are poor if the input-output characteristics are not consistent in both training and testing phases.
integrated artificial neural networks (ANNs) with K-means clustering method, called clustering-based hybrid inundation model (CHIM), to forecast 1-h-ahead inundation extents and depths.Leedal et al. (2010) proposed twodimensional (2-D) real-time probabilistic inundation maps based on a modified Kalman filter model coupled into 2-D hydrodynamic model to compute the maximum multistepahead inundation extent.Pan et al. (2011) used hybrid ANNs in rainfall-inundation forecasting to estimate 1-h-ahead inundation depths.

Fig. 1 .
Fig. 1.Architecture of the T-NARX network during training and testing phases in the time delay mode.

Fig. 2 .
Fig. 2. Architecture of the O-NARX network during training and testing phases in the online time delay mode.

Fig. 3 .
Fig. 3. Architecture of the R-NARX network during training and testing phases in the recurrent mode.

Fig. 6 .
Fig. 6.Comparison of simulated and (a) one-hour-ahead, (b) three-hour-ahead, (c) six-hour-ahead forecasted inundation depths of the O-NARX and R-NARX models during the testing phase at S2.

Fig. 7 .
Fig. 7. Stability comparison of the O-NARX and R-NARX models during the forecast period of 24 h (the mean RMSE of thirteen sites).

Table 1 .
24-h cumulative rainfall of design rainfall events at three rainfall gauges.
*All 24 design events were used for training data set.

Table 2 .
Rainfall information of the 31 historical rainfall events.
* Two events are used for training data (+) and the other 29 events are used for testing data.

Table 3 .
Maximum, mean and variance of the 13 sites' flood depths.

Table 4 .
Performance of one-to six-hour-ahead flood depth forecasts of the T-, O-and R-NARX models for all thirteen sites