The Bacchiglione catchment (north-eastern Italy, see Fig. 1) is one of the case studies in which the WeSenseIt (WSI) Citizen Observatory of Water project (http://wesenseit.com) developed and tested innovative static and low-cost mobile sensors (Ciravegna et al., 2013). The main goal of the WSI project was to allow active citizens to support the work of water authorities by providing CS observations. Innovative static sensors were strategically integrated into the existing monitoring networks for collecting physical and CS data. Low-cost mobile sensors were developed such as a mobile phone application, which uses a quick response (QR) code for geographical referencing and allows to send, among others, flood reports and water level (WL) observations. In addition, the WSI project set up a pilot platform in which CS observations collected with this application can be sent. However, this pilot is not yet operational and CS observations are not yet available (see details of the testing of this pilot in Sect. 2.3). In this research, only WL data are assimilated.

This research focuses on the upper part of the Bacchiglione catchment which flows into the Adriatic Sea in the south of the Venetian Lagoon. The case study has an overall extent of about 450 km2 with a river length of approximately 50 km. The three main tributaries are the Timonchio River on the east side and Leogra and Orolo rivers on the west side. The main urban areas are located close to the outlet section of the case study area, the city of Vicenza. The Alto Adriatico Water Authority (AAWA) is currently using an operational semi-distributed hydrological and hydraulic model for early warning (Ferri et al., 2012, Mazzoleni et al., 2017a). Forecasted and measured precipitation time series are available for a flood event that occurred in May 2013. The forecasted precipitation time series are provided by the COSMO-LAMI model, a regional model that provides numerical prediction over the national territory at 7 km resolution and 3-day time interval. Currently, AAWA is performing quality control on the forecasted data before using them in the Bacchiglione flood early-warning system. The measured precipitations are supplied and validated by Veneto Regional Agency of Environmental Prevention and Protection (ARPAV). The event of May 2013 is considered to be significant due to its high intensity, which resulted in several traffic disruptions at various locations upstream of Vicenza. In this study, we assess the usefulness of assimilating CS WL (synthetic) observations in the hydrological and hydraulic models to improve model performance and consequently flood prediction.

Although CS observations were neither operational nor available in the case study, we analysed the characteristics of each sensor to generate the synthetic WL observations that we assimilated for the flood event of 2013. We considered three types of sensors to measure WL, static physical, static social and dynamic social sensors. Currently, only StPh sensors are used by AAWA to provide daily flood forecasts in the Bacchiglione catchment. This section of the paper aims to describe the characteristics of these sensors in terms of spatial coverage and accuracies.

The StPh sensors are traditional physical sensors such as water level ultrasonic sensors. StPh have a fixed location and a regular measurement interval. Data from StPh sensors are validated by ARPAV. Observational error depends on how well the cross section where the StPh sensor is located is documented and on random and bias errors due to sensor characteristics. Despite the potential observational error, we assume a high accuracy level as the observation is automatically generated by the sensor and therefore not affected by the variability of CS data.

StSc sensors have a higher spatial distribution than StPh sensors along the river reach but are characterized by intermittent CS observations. The StSc sensors are staff gauges at safe, strategic and accessible locations along the river reaches. Citizens can report observations using these static sensors to estimate WL values. According to the data collection tool, CS observations can come in a variety of formats either quantitative or qualitative, which is often one of the biggest challenges when involving citizens. Automatic mechanisms for data processing can be implemented. For example, whenever photos are collected, these can be automatically analysed using image recognition methods as proposed by van Overloop and Vierstra (2013) and Le Boursicaud et al. (2016). In this case, a reference gauge must be available. The WSI mobile phone application will be used to send quantitative measurements (water level) observed at a specific staff gauge. Photos and videos are not supported by the WSI application. The geographical referencing will be provided by means of QR codes together with associated date and time. The WSI mobile application is equipped with a filter that automatically discards those water level measurements that fall outside the range associated with the staff gauge.

DySc sensors do not have fixed locations. Water level observations at a particular location via a smartphone application can be requested by water authorities according to the accessibility of the location. A possible method for measuring flow using DySc sensors is described in Lüthi et al. (2014). The authors proposed an approach based on particle image velocimetry to estimate with acceptable accuracy water level, surface velocity and runoff in open channels. However, this approach requires a priori knowledge of the channel geometry at the location of the measurement, which is one of the main sources of uncertainty. For this reason, in this paper it is assumed that DySc sensors have lower accuracy than StSc sensors. Another example of DySc sensors is reported in Michelsen et al. (2016) where water level time series are derived from the analysis of YouTube videos. It is worth noting that the WSI mobile application does not allow for automatic retrieval of flow information from photos and video as proposed in Lüthi et al. (2014).

As reported in Table 1, WL observations have different characteristics of temporal availability and accuracy based on the adopted sensor and changes in the cross section. Regardless of the type of social sensor, whether expert or amateur, we acknowledge that the data accuracy and intermittency of CS observations can be affected by various factors. Source of errors in observations include but are not limited to the following (Cortes Arevalo, 2016; Kerle and Hoffman, 2013; Le Coz et al., 2016): (i) the expertise level (training and experience is required to read a gauge, take a picture and use the mobile application), (ii) type and format of CS observation based on sensor classification and data collection procedure (WL measurement and photo with reference to a staff gauge vs. a photo with reference to a neighbouring object), and (iii) the specific conditions at the reporting location (accessibility, visibility and environmental conditions). Intermittency (temporal availability) of the CS observations is directly related to CIL, i.e. the probability of receiving a CS observation. In addition, CS observations imply the filtering and integration of a variety of formats and information types, which are required to develop suitable tools for data collection and processing (Kosmala et al., 2016).

Gharesifard and Wehn (2016) categorized participants into “netizens”, citizen scientists and volunteers to accordingly distinguish: (i) unawareness about their implicit involvement and contribution to monitoring networks (netizens); (ii) explicit and intentional involvement in data provision (citizen scientists) and (iii) the involvement of individuals or groups that are systematically targeted and recruited to participate in data provision with predefined goal(s) (volunteers).

In the framework of the WeSenseIt project, an exercise was carried out with volunteers who were providing water level observations via the smartphone application, from a limited number of locations to test the pilot set up. However, due to the limited number of participants, duration and testing goal of the exercise, no formal assessment of citizen involvement could be undertaken. For this reason, we propose theoretical involvement scenarios to represent the hypothetical situations according to which citizens are fully or partially involved in the Bacchiglione catchment. In the numerical simulations performed in this study, we did not make a distinction between citizen expertise (expert or amateur) and involvement type (citizen scientists or volunteers). We do not refer to the engagement process (how to get citizens involved) but rather to the involvement level (probability of receiving a CS observation based on the citizen's own interest or intention in collecting water levels). In fact, motivations and involvement levels are the only variables that differentiate the citizens, as described in the next sections.

In order to implement the semi-distributed model, the Bacchiglione catchment is divided into different sub-catchments and the so-called inter-catchments which contribute streamflow to the main river channel up to the urbanized area of Vicenza. In the schematic representation of the Bacchiglione catchment (see Fig. 1), the location of the StPh and StSc sensors corresponds to the outlet section of the three main sub-basins, Timonchio, Leogra and Orolo. The remaining sub-basins are considered as inter-catchments. The rainfall–runoff processes within each sub-catchment and inter-catchment are represented by the conceptual hydrological model developed by AAWA. In the case of the main river channel, a hydraulic model is used to propagate the flow down to the gauge station of PA in Vicenza. The river reach is divided into several reaches according to the location of the internal boundary conditions. We use hydrological outputs as upstream (from sub-catchments) and internal boundary conditions (from inter-catchments). Figure 1 shows that the output of the hydrological model (red arrows) are boundary conditions for the proposed hydraulic model.

The Kalman filter (KF, Kalman, 1960) is a mathematical tool widely used to integrate real-time noisy observations, in an efficient computational (recursive) algorithm, within a dynamic linear system resulting in the best state estimate with minimum variance of the model error. In Liu et al. (2012), a detailed review of KF and other types of data assimilation approaches is reported. The first step in the KF procedure is the forecast of the model state vector, following Eq. (4), and the covariance matrix is expressed as follows: Pt-=ΦPt-1+ΦT+St, where the superscript “–” indicates the forecasted model error covariance matrix P and the superscript “+” indicates the updated state value coming from the previous time step. When an observation zo becomes available, the second (update) step of the KF is executed, in which the forecasted model states x and covariance P are updated as follows: xt+=xt-+Ktzto-Htzto,Pt+=I-KtHtPt-,Kt=Pt-HtTHtPt-HtT+Rt-1, where K is the Kalman gain matrix (the higher its values, the more confidence KF gives to the observation zo and vice versa). Due to the fact that along the river channel only WL observations are provided, the Manning equation is used to express the vector z0 as streamflow based on the river cross-sectional geometry.

In this study, CS observations are considered. As already mentioned, such observations can be irregular both in time and in space. In order to consider the intermittent nature in time within the KF, the approach proposed by Cipra and Romera (1997) and Mazzoleni et al. (2015) is adopted. According to this approach, when no observation is available, the model state vector x is estimated using Eq. (4), while the model error covariance P is left unchanged: Pt+=Pt-. It is worth noting that in the case of a hydraulic model, the state variables at each reach are updated independently.

In operational practice, WL values are converted into streamflow values to be then assimilated within hydrological models. This is usually done using the available rating curves at the sub-catchment outlets. However, WL data can usually be directly assimilated in hydraulic models, but the problem is that the MC model used in this study requires flow information rather than WL. For this reason, the synthetic WL observation at a certain random location (DySc sensor) is converted into streamflow by means of the Manning equation if no rating curve information is available. In fact, it is quite unlikely to have the information of the rating curve at a random location provided by DySc sensors in real-world applications. When there are no data regarding the cross section, assumptions should be made about a rectangular cross section with a given width and depth. However, this approach will introduce significant uncertainty in river flow estimation. A possible solution is the use of mobile applications able to automatically retrieve flow information from photos and video as proposed in Lüthi et al. (2014), Overloop and Vierstra (2015) and Le Boursicaud et al. (2015). We believe that these types of mobile applications will become increasingly available (at reasonably low costs) to citizens in order to easily measure river flow.

Due to the lack of distributed CS observations at the time the considered flood event occurred, synthetic WL observations are used (Mazzoleni et al., 2017a). In order to generate these synthetic observations, the observed time series of precipitation during the considered flood event are used as input for the hydrological models of the sub-catchments and inter-catchments to generate synthetic discharges and then propagate them with the hydraulic model down to the prediction point of PA (corresponding to the sensor StPh-3 in Fig. 1). In this way, the synthetic WL values at the outlet of the sub-catchments or inter-catchments and at each spatial discretization of the six reaches of the Bacchiglione River are estimated, and assumed as observed variables in the assimilation process. In meteorology, this kind of approach is often called an “observing system simulation experiment” (OSSE), as described for example by Arnold and Dey (1986), Errico et al. (2013) and Errico and Privé (2014).

Regarding the observation error, as described in Weerts and El Serafy (2006), Rakovec et al. (2012), and Mazzoleni (2017), the covariance matrix R is assumed to be as follows: Rt=αt⋅Qtsynth2, where α is a variable related to the accuracy level of the measurement. The accuracy (i.e. degree to which the measurement is correct overall) is subjected to random error and bias or systematic errors (Bird et al., 2014). Moreover, for WL observations, accuracy levels vary temporally, spatially, and for each physical or social sensor. Table 2 summarizes the distribution of the coefficient α of the observational error of Eq. (12). The distribution of the coefficient α does not pretend to be exhaustive in accounting for the different accuracies between observations coming from physical and social sensors, but a first and simplified approximation that is a possible aspect for further research (see details in Sect. 2.2 and Table 1).

Although there are many sources of uncertainty in the indirect estimation of streamflow, for StPh sensors it is assumed that the rating curve estimation is the main source of uncertainty to properly estimate the streamflow given a certain WL value. In fact, for the StPh sensors used in this study the instrument precision is about 0.01 m. As described in Weerts and El Serafy (2006) and Rakovec et al. (2012), the coefficient α is assumed equal to 0.1, constantly in time and space.

However, due to the unpredictable accuracy of the CS observations coming from the StSc and DySc sensors, the coefficient α is assumed to be random stochastic, variable in time and space within a minimum (αmin) and maximum (αmax) value, and based on the type of sensor and citizen accuracy. Table 2 summarizes the values for the accuracy levels that are used in this study and are assumed under the following considerations:

For both StSc and DySc, sensor α values are higher than those of StPh sensors due to the additional sources of uncertainty introduced with the CS WL estimation and the consequent conversion to discharge. Moreover, the coefficient α for both StSc and DySc sensors is considered to be a random stochastic variable uniformly distributed in time and space (see Table 2).

In the first experiment, CS observations are taken from StSc (experiment 1.1) and DySc (experiment 1.2) sensors according to random CILs. Such involvement, closely related to the intermittent nature of the WL observations, can be considered as the probability of receiving an observation at a given model time step. This means that in the case of CIL = 0.4 there is 40 % of probability of obtaining an observation at a given model time step. In fact, in the case of CIL = 0, no observation is assimilated and the semi-distributed model is run without any update, whereas if CIL = 1, observations are available at every time step and this situation is analogous to the observation from StPh sensors, which are assumed to be regular in time.

In this experiment, all the StPh, StSc and DySc sensors are considered. One main problem in citizen science is understanding the motivations that drive citizens to be involved in such activities (Gharesifard and Wehn, 2016). For this reason, a theoretical assumption about citizen involvement based on their motivations, varying in time and space, is introduced. In the previous experiments, involvement is considered to be random varying from 0 to 1. In this experiment, involvement level is assumed to be a function of the spatial distribution of the population within the Bacchiglione catchment.

As stated by Gharesifard and Wehn (2016), we acknowledge that stronger motivations or intentions are not only driven by a combination of more positive and favourable attitudes. The motivations also rely on stronger positive social pressure and greater perceived control or self-sufficiency regarding the means to provide CS observations. In this paper, the distinction between favourable attitudes are treated from a theoretical point of view since during the WSI project, no consistent analysis of motivational structures was undertaken for the Bacchiglione case study. Based on Batson et al. (2002), we assume the three main motivations for citizens involvement in collecting data: (1) for their own personal purposes (usefulness of the collected data for personal interest or direct flood risk management impact), (2) belonging to a community of peers with shared interests and (3) altruism (benefiting society at large). In order to assess citizen involvement, we propose a three-step procedure consisting of (1) estimation of the active-citizen area; (2) estimation of the number of active citizens and (3) estimation of the citizen involvement curve.

Step 1 involves the estimation of the “active-citizen area”. A hypothetical 500 m buffer around each sub-river reach of 1000 m (spatial discretization of the MC model) is used to identify the area in which the active population might provide CS observations using DySc sensors (see Fig. 2). It is assumed that the citizens located further than 500 m from the river are not contributing to the collection of CS observations. In the case of the StSc sensor, we assume the active area to be a circle with a 500 m radius with the sensor at the centre. Different extents of the buffer will lead to different coverages of the active area, with significant effects on the simulated number of hypothetically involved citizens. However, analysing the implications of different buffer extents on the number of active citizens and subsequent flood predictions is out of the scope of this research. Land cover maps are used to identify the main urban area from which citizens might provide CS observations of WL within the buffer previously estimated (see Fig. 2).

Step 2 involves the estimation of the number of active citizens. The population density for the different municipalities along the different river reaches is used to estimate the number of citizens within the 500 m buffer of each sub-river reach in which the urban areas are located. In the case of agricultural areas, an involvement value equal to zero is considered. In addition, not all citizens would be able to provide CS observations because only a certain proportion of them use mobile phones. According to Statistica (2016), the mobile phone market penetration in Italy in 2013, the year of the flood event analysed in this study, was about 41 %, which means that about 41 % of the population was potentially able to submit data. In view of the lack of a better source, we assume that this proportion is also valid for the regional scope. Therefore, to estimate the potential number of active citizens that could submit data close to the river reach, we first estimate the total population enclosed in a cell of 1 km long by 1 km wide (a buffer of 500 m from each side of the river) and then estimate 41 % of this. Table 4 summarizes the results for the case of the StSc sensors and Table 5 those for the DySc sensors. In Table 5, the active citizens are divided by the number of sub-reaches (3 for reach 6). For reach 6 (at kilometers 3, 4, and 5), the main urban areas are contained in more than one sub-reach. Naturally, for a better estimation of these values, a more exhaustive social–economic analysis should be performed.

Step 3 involves the estimation of the theoretical citizen involvement curve. It is now necessary to estimate the level of citizen involvement based on the hypothetical number of active citizens and their motivation for sharing data. For this reason, three different involvement curves, each representing a scenario and corresponding number of active citizens, providing the maximum citizen involvement level (MCIL), are proposed. These scenarios are based on Batson et al. (2002), whose aggregated categories of citizen's motivations are still in agreement with more comprehensive and detailed analyses such those recently reported in Geoghegan et al. (2016) and Gharesifard and Wehn (2016).

In scenario 1, we assume that citizens collect data mainly for their own personal use. In this case, the MCIL is low for a low number of citizens, while it grows following a logistic function, Eq. (15), for increasing numbers of people. MCIL=K⋅Po⋅er⋅PopK+Po⋅er⋅Pop-1+w, where Pop is the population; r is the growth rate (we assumed two different values of r, 0.04 and 0.08); K is the carrying capacity, i.e. the maximum value of MCIL, assumed to be equal to 1; w is a coefficient related to the additional CS observations received from enthusiastic individuals (third citizen scenario explained below); and Po is the minimum value of MCIL assumed equal to 0.01.

In scenario 2, citizens might decide to collect and share CS observations driven by a feeling of belonging to a community of peers with shared interests and vision. In this case, it is assumed that a maximum value of MCIL is achieved for small population values while for increasing population this value is decreasing. This scenario follows an inverse logistic function as shown in the graphical representation of scenario 2 in Fig. 3.

In scenario 3, enthusiastic individuals might provide additional information driven by moral norms and the wish to create knowledge about the hydrological status of the river, benefiting society at large. This is potentially a much smaller subset of the population. The added value of this information is accounted for in Eq. (15) by means of a coefficient w. Table 6 summarizes the different involvement curves based on the previous scenarios and different values of the coefficients r and w.

In the next phase of analysis, a number of model runs (100) are carried out, considering the random values of citizen involvement from 0 to the MCIL according to the given involvement scenarios and the population. For example, considering scenario 1 and 150 inhabitants enclosed in a given river sub-reach, several model runs are performed for involvement values varying from 0 to 0.65 based on Fig. 3. In case different CS observations are coming in at the same time from different sensors, only the most accurate observation, i.e. that having the lower value of the coefficient α in Eq. (12), is assimilated in the hydrological and/or hydraulic model. Another approach could be to assimilate all measurements instead of only the most accurate ones. In this case, each observation is used within the assimilation scheme with the account of its error: less weight would be given to the more uncertain observations.

Finally, this experiment also investigated the effect of the spatial variability of smartphone market penetration and decrease in citizen involvement levels over time. For this reason, a higher (double) percentage of active citizens in Vicenza is assumed (smartphone market penetration of 80 %), while random values of the coefficient r are considered to represent lower involvement levels over time.

Experiment 2 focuses on the assimilation of CS observations from a distributed network of heterogeneous StPh, StSc and DySc sensors. In particular, the involvement level is calculated in a more realistic way, accounting for the population living in the range of 500 m from the river. Based on Fig. 3, different MCIL values are calculated for the three scenarios in collecting and sharing WL observations. It is worth noting that bias 2 is considered in the CS observations from DySc sensors.

Figure 8 shows μ(NSE) values in the case of different involvement scenarios and MCIL according to the different types of sensors. A random value of involvement level between 0 and MCIL is considered for a given river sub-reach and model run. In particular, in Fig. 8, smaller values of MCIL such as MCIL1, MCIL2, MCIL3, MCIL4 and MCIL5 are estimated as 0.2 MCIL, 0.4 MCIL, 0.6 MCIL, 0.8 MCIL and MCIL, respectively. Note that scenario 2 is the one providing the best model improvements, followed by scenario 3. Involving the enthusiastic people (scenario 3) helps to improve μ(NSE), especially for low involvement values. Scenario 1 is the one that gives the lowest μ(NSE) values due to the lowest growth rate of the involvement curve and consequent lower involvement of citizens.

In scenarios 1 and 3, the steepest vertical gradient of the contour plot can be observed, leading to the conclusion that model results seem to be more sensitive to the change in MCIL values in StSc sensors rather than DySc sensors. However, the gradient reduces with scenario 2.

In the previous analysis, NSE is used as the only performance indicator without considering improvement in the prediction of the peak and rising limb of the hydrograph, which are extremely important in operational flood management. For this reason, the relative error between the observed streamflow peak and simulated peak (see Eq. 16) is included to better assess the assimilation of crowdsourced observations from an operational point of view. ERR=WL,PO-WL,PSWL,PO, where WL,PO and WL,PS are the observed and simulated streamflow (m3 s-1). The results reported in Fig. 8 show comparable results to those achieved using NSE. Including CS observations from enthusiastic citizens seems not to lead to a more accurate representation of the peak discharge. In fact, similar μ(NSE) values are achieved between scenarios 1 and 3. However, error in peak prediction is lower in scenario 1 than in scenario 2. It can be observed that ERR values are clearly more sensitive to the different involvement values in StSc sensors than DySc sensors (vertical gradient).

In the previous analysis, unrealistically high citizen involvement (up to 80 %) is considered. For this reason, the following analysis focuses more on the lower part of the theoretical involvement curve, assuming more realistic CIL. In particular, the maximum carrying capacity of the logistic curve (K) is changed from 0.01 up to 1. In the case of K equal to 1, the values of μ(NSE) related to the different scenarios are estimated as mean average of the contour plot shown in Fig. 8. The same analysis is performed for the vector of different values of K.

The results of this analysis show an expected reduction in the model performances for low values of the parameter K (which indicates the maximum possible level of involvement). It can be noted that if K is equal to 0.5, assimilation of crowdsourced observations still provide significant model improvement for all the different scenarios even though the involvement is halved. As expected, σ(NSE) values tend to increase for low involvement of citizens. From Fig. 9, it can be seen that μ(NSE) values do not follow a linear trend as expected. On the contrary, it tends to drop for values of K between 0 and 0.2 (for example in scenario 3), while for higher K values μ(NSE) does not grow significantly. In particular, for K values higher than 0.5, scenario 2 provides the highest μ(NSE) values. Besides, for K values lower than 0.5, scenario 3 is the one leading to better model performances. This is because the presence of enthusiastic individuals keeps high involvement values even for low values of K. Regarding the variability of NSE, i.e. σ(NSE), for values of K lower than 0.4, high σ(NSE) can be observed in scenario 1.

Additional analysis considering negative and positive bias (bias 3 and 4 in Table 3) in the CS observations are considered (see Fig. 10). As expected, it can be observed that bias 4 provides higher NSE values than bias 2 since the model without update underestimate observed streamflow or water level. Moreover, results obtained using observations with bias 3 have lower NSE than the results with bias 2. However, in both bias 3 and 4, such changes in NSE are very small, leading to the conclusion that assimilation of biased WL observations during the May 2013 flood event in the Bacchiglione River do not reduce model performances.