<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing with OASIS Tables v3.0 20080202//EN" "journalpub-oasis3.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:oasis="http://docs.oasis-open.org/ns/oasis-exchange/table" dtd-version="3.0"><?xmltex \makeatother\@nolinetrue\makeatletter?>
  <front>
    <journal-meta>
<journal-id journal-id-type="publisher">HESS</journal-id>
<journal-title-group>
<journal-title>Hydrology and Earth System Sciences</journal-title>
<abbrev-journal-title abbrev-type="publisher">HESS</abbrev-journal-title>
<abbrev-journal-title abbrev-type="nlm-ta">Hydrol. Earth Syst. Sci.</abbrev-journal-title>
</journal-title-group>
<issn pub-type="epub">1607-7938</issn>
<publisher><publisher-name>Copernicus Publications</publisher-name>
<publisher-loc>Göttingen, Germany</publisher-loc>
</publisher>
</journal-meta>

    <article-meta>
      <article-id pub-id-type="doi">10.5194/hess-21-153-2017</article-id><title-group><article-title>Practitioners' viewpoints on citizen science in water management: <?xmltex \hack{\newline}?>a case
study in Dutch regional water resource management</article-title>
      </title-group><?xmltex \runningtitle{Practitioners' viewpoints on citizen science in water management}?><?xmltex \runningauthor{E.~Minkman et al.}?>
      <contrib-group>
        <contrib contrib-type="author" corresp="no" rid="aff1 aff2 aff3">
          <name><surname>Minkman</surname><given-names>Ellen</given-names></name>
          
        </contrib>
        <contrib contrib-type="author" corresp="no" rid="aff2">
          <name><surname>van der Sanden</surname><given-names>Maarten</given-names></name>
          
        </contrib>
        <contrib contrib-type="author" corresp="yes" rid="aff1">
          <name><surname>Rutten</surname><given-names>Martine</given-names></name>
          <email>m.m.rutten@tudelft.nl</email>
        </contrib>
        <aff id="aff1"><label>1</label><institution>Department of Water Resource Management, Delft University of
Technology, Delft, 2628 CN, the Netherlands</institution>
        </aff>
        <aff id="aff2"><label>2</label><institution>Department of Science Education and Communication, Delft University of
Technology, Delft, 2628 CJ, the Netherlands</institution>
        </aff>
        <aff id="aff3"><label>3</label><institution>Presently at Department of Public Administration and Sociology,
Erasmus University Rotterdam, Burgemeester Oudlaan 50, 3062 PA Rotterdam,
the Netherlands</institution>
        </aff>
      </contrib-group>
      <author-notes><corresp id="corr1">Martine Rutten (m.m.rutten@tudelft.nl)</corresp></author-notes><pub-date><day>9</day><month>January</month><year>2017</year></pub-date>
      
      <volume>21</volume>
      <issue>1</issue>
      <fpage>153</fpage><lpage>167</lpage>
      <history>
        <date date-type="received"><day>3</day><month>September</month><year>2016</year></date>
           <date date-type="rev-request"><day>19</day><month>September</month><year>2016</year></date>
           <date date-type="rev-recd"><day>29</day><month>November</month><year>2016</year></date>
           <date date-type="accepted"><day>30</day><month>November</month><year>2016</year></date>
      </history>
      <permissions>
<license license-type="open-access">
<license-p>This work is licensed under a Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/3.0/">http://creativecommons.org/licenses/by/3.0/</ext-link></license-p>
</license>
</permissions><self-uri xlink:href="https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017.html">This article is available from https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017.html</self-uri>
<self-uri xlink:href="https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017.pdf">The full text article is available as a PDF file from https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017.pdf</self-uri>


      <abstract>
    <p>In recent years, governmental institutes have started to use citizen science
as a form of public participation. The Dutch water authorities are among
them. They face pressure on the water governance system and a water awareness
gap among the general public, and consider citizen science a possible
solution. The reasons for practitioners to engage in citizen science, and in
particular those of government practitioners, have seldom been studied. This
article aims to pinpoint the various viewpoints of practitioners at Dutch
regional water authorities on citizen science. A Q-methodological approach
was used because it allows for exploration of viewpoints and statistical
analysis using a small sample size. Practitioners (33) at eight different
water authorities ranked 46 statements from agree to disagree. Three
viewpoints were identified with a total explained variance of 67 %.
Viewpoint A considers citizen science a potential solution that can serve
several purposes, thereby encouraging citizen participation in data
collection and analysis. Viewpoint B considers citizen science a method for
additional, illustrative data. Viewpoint C views citizen science primarily as
a means of education. These viewpoints show water practitioners in the
Netherlands are willing to embrace citizen science at water authorities,
although there is no support for higher levels of citizen engagement.</p>
  </abstract>
    </article-meta>
  </front>
<body>
      

      <?xmltex \hack{\newpage}?>
<sec id="Ch1.S1" sec-type="intro">
  <title>Introduction</title>
      <p>The OECD (Organization of Economic Cooperation and Development) named the
Netherlands “an international example” of water resource management in
their 2014 report, but warns against “a striking awareness gap among Dutch
citizens related to key water management functions, how they are performed
and by whom” (OECD, 2014, p. 21). The main causes of this awareness gap are
the absence of major water calamities in the past 60 years and the
improvement of water quality over the past decades. Dutch citizens take the
excellent water resource management for granted (OECD, 2014), which poses
social challenges for Dutch water resource management. Citizens' behaviour
counteracts efforts of water authorities; flood defences are violated by
property development and civic pollution is common (OECD, 2014). Citizens and
interest groups do not recognize water threats (Tielrooij, 2000), which
causes a decreasing support for investments in flood defence and water
quality management (OECD, 2014; Tielrooij, 2000; UvW, 2015a). Other countries
also experience a “lack of citizen concern about water policy and low
involvement of water users' associations” (OECD, 2011, p. 60). Half of the
reviewed OECD countries across the globe face such challenges, including
Chile, Italy, Korea and Mexico (OECD, 2011, p. 61). The Dutch water
authorities (Unie van Waterschappen, UvW) governing
body
concluded that collaboration with other government layers, industry, interest
groups and citizens is needed (UvW, 2015a). The UvW envisions increased
public participation, with citizen science as a form of such participation
(UvW, 2015b). In addition to raising awareness, citizen science could
contribute to data collection and help water authorities to enhance their
monitoring programmes, particularly with respect to the
Water Framework Directive.</p>
      <p>Definitions of citizen science can be narrow and focussed solely on data
collection for academic purposes. Silvertown (2009) describes citizen science
in a broader perspective, applicable to citizen science in practitioners'
activities considered in this article. “Today, most citizen scientists work
with professional counterparts on projects that have been specifically
designed or adapted to give amateurs a role, either for the educational
benefit of the volunteers themselves or for the benefit of the project. The
best examples benefit both” (Silvertown, 2009, p. 467). To prevent
confusion, a distinction is made within this category of professional
counterparts (Silvertown, 2009). The professional counterparts in
Silvertown's definition include scientists, conservation professionals and
government practitioners. We define scientists as those involved in academia.
Conservation professionals are those working for nature managers or
conservation organizations. Government practitioners are defined as those
working at a government agency or at the local government level.</p>
      <p>Citizen science in water resource management is upcoming but lingering on the
verge of a breakthrough (Buytaert et al., 2014; Cohn, 2008; Fraternali et
al., 2012). The rise of robust, cheap and low-maintenance sensors enhances
opportunities for citizen science in the complex arena of water resource
management (Buytaert et al., 2014). New hydrological modelling frameworks
(e.g. Clark et al., 2015) and specifically uncertainty quantification (e.g.
Shoaib et al., 2016) can allow for more effective design of citizen science
campaigns to systematically reduce model uncertainty, as well as more
effective use of citizen-collected data. These data are often considered
highly uncertain by practitioners, as they indicated in the exploratory
interviews held at the start of this research. Fraternali et al. (2012) give
an overview of the potential of amateurs taking part in data collection, data
analysis and the process of decision-making in water resource management.</p>
      <p>This potential is also recognized in the Netherlands. In November 2014 water
authority Delfland organized a workshop<fn id="Ch1.Footn1"><p>Part of the symposium “De
fysieke Digitiale Delta” (the Physical Digital Delta; see also
<uri>http://www.digitaldelta.nu/en/events/item14</uri>).</p></fn> on big data and citizen science in Delft, the Netherlands.
Dutch regional water managers expressed their interest in citizen science
during this workshop, although they also indicated their doubts regarding
this approach. Their main questions were (a) “what motivates citizens to
participate?”, (b) “what should be the role of citizens?”, and (c) “why
should a water authority engage in citizen science?”</p>
      <p>Citizens' motivations have been studied extensively in a diverse set of
citizen science projects, such as online crowdsourcing (e.g. Chandler and
Kapelner, 2013; Raddick et al., 2010; Rogstadius et al., 2011), environmental
monitoring (e.g. Hobbs and White, 2012; Roy et al., 2012) and meteorology
(e.g. Gharesifard and Wehn, 2016). It has been acknowledged that the idea of
“the public” does not exist (e.g. Varner, 2014), since “the public”
consists of a wide variety of people with different backgrounds, interests,
traits, values and beliefs. Nevertheless, existing studies of citizen science
in field and online projects, despite this diversity, reveal the same
dominant motivations over a wide range of projects and participants. The most
mentioned reasons for citizens to engage in citizen science are because they
think it is fun, because the topic interests them, and because the topic
matters to them, e.g. they want to contribute to science or nature
conservation (e.g. Chandler and Kapelner, 2013; Hobbs and White, 2012;
Raddick et al., 2010; Rogstadius et al., 2011; Roy et al., 2012). Citizens
are motivated to continue to contribute by (increasing) the extent of their
involvement (Rotman et al., 2012; Roy et al., 2012), offering feedback
concerning the work at three levels (individual contribution, group
contribution and the use of data) and building a relationship based on trust
between scientists and citizens (Rotman et al., 2012). The importance of
trust is stressed by authors in the field of water management as well
(Buytaert et al., 2014; Gharesifard and Wehn, 2016).</p>
      <p>The role of citizens varies depending on the purpose of the citizen science
project. Tulloch et al. (2013) studied the purpose of citizen science
projects in bird watching, a research field with a century-long history of
citizen science. The most common purpose of citizen science is knowledge
generation (knowledge also new to the involved professionals), followed by
improving monitoring methods and raising awareness among citizens. In
water resource management, citizen science can enhance knowledge about the
water system, for example by generating knowledge on different spatial or
temporal scales. It may also be used to add or test new monitoring methods
for the existing monitoring network (mentioned in exploratory interviews held
at the start of this study). Citizen science can also be used to increase the
water awareness that was dubbed absent in the OECD report (OECD, 2014).
According to Tulloch et al. (2013), citizen science can be used to improve
management practice as well. In water resource management such improved
management can be the result of more frequent monitoring. The literature
mentions public education as an important purpose of citizen science (e.g.
Cohn, 2008), but Tulloch et al. (2013) found that public education was rarely
the main purpose. Other identified, yet also more rare, purposes include
doing social research (e.g. on human behaviour), offering recreation and
serendipity (i.e. unexpected discoveries). A more recent study in the field
of ecology specified the potential of citizen science for the purpose of
policy development (Hollow et al., 2015). In early stages of a policy
development process, citizen science can be used to discover alternative
management actions. In case there is a range of alternatives, citizen science
can be used to gauge public opinion. In later stages it can be used to
persuade public opinion towards a desired alternative or to provide a legal
justification for the chosen policy. Citizen science-based data can be used
for decision-making in water resource management as well (Macknick and
Enders, 2012). Projects can have one or multiple purposes, as the iSPEX
project demonstrates. This project served knowledge generation, public
education and method improvement (Land-Zandstraet al., 2015; Snik et al.,
2014).</p>
      <p>For the purposes of knowledge generation, improving monitoring methods
improves management and policy development. Bonney et al. (2009) provide a
useful classification of citizens' roles. They suggest there are basically
three levels of citizen involvement possible: contribution, collaboration and
co-creation. In a contributory project, citizens are mainly involved in
data collection; the research question and design are done by scientists or
experts. In collaborative projects citizens are involved in the analysis
and can be involved in the design and dissemination of results as well. In
co-created projects citizens are involved in all steps of the research
process and may even initiate the project. The vast majority of studies in
the overview presented by Bonney et al. (2009) considered contributory
projects. Even the occasionally co-created projects were part of multi-case
studies in which contributory projects dominated the results. Citizens'
involvement in activities other than data collection may serve different
purposes; for example, citizen-based goal-setting could enhance
adaptive management practices (Cooper et al., 2007). Bonney et al. (2009) is
frequently cited (e.g. Rotman et al., 2012; Roy et al. 2012) and can be
considered a typical classification. The levels of involvement align to a
large extent with the governance structures defined by Conrad and
Hilchey (2011): consultative, collaborative and transformative governance. In
transformative governance citizens initiate a project. An example can be
found in the global community monitor that measures air and water quality on
a global scale (Conrad and Hilchey, 2011).</p>
      <p>For the purposes of public education, raising awareness, improving management
and policy development, science communication literature (e.g. Varner, 2014)
provides a more useful classification of the interaction between citizens and
professional counterparts. We use the term science communication here in a
broad sense and include communication between public and all professional
counterparts referred to by Silvertown (2009), including practitioners.
Citizen science is often viewed as a form of informal science education,
contributing to public awareness of science (PAS) and public understanding of
science (PUS). We think this view is too limited, as it only encompasses the
deficit model of science communication. Higher levels of involvement public
engagement of science (PES) and public participation in science (PPS) are
possible (Van der Auweraert, 2005), particularly in the collaborative and
co-created projects in Bonney's definition. An example of such participation
is the water quality programme in Rhode Island mentioned in the review by
Conrad and Hilchey (2011).</p>
      <p>The motivation of professional counterparts to engage in citizen science has
been less frequently studied than the motivation of citizens, and to our best
knowledge research on professionals' motivation is limited to scientists.
Scientists' motivations are primarily to advance science as well as develop
their careers (Rotman et al., 2012). This is aligned with citizens'
motivational desire to contribute to science and conservation or to engage in
exploring a topic of their interest further (e.g. Rotman et al., 2012).
Weng (2015) identifies three areas of friction between the vision of
scientists and volunteers with regard to citizen science. The first area of
friction is often short-term participation of volunteers that conflicts with
scientists' interest in long-term processes. The second area concerns the
limits of what volunteers can do and their dissatisfaction with the research
processes. The third area regards a power hierarchy between citizens and
scientists. Rotman et al. (2012) found that while the motivations of citizens
and scientists are complementary, they can also change over time. Therefore,
continued attention by those who are managing citizen science projects with
regard to matching these motivations is crucial.</p>
      <p>The motivation of scientists cannot be translated one-to-one to practitioners
or (local) government representatives for two reasons. First, scientists are
concerned with scientific data collection (Rotman et al., 2012), while
practitioners are often interested in improving management practices (Weng,
2015) and government agencies are concerned with policy-making (Hollow et
al., 2015). Second, the different role of authorities leads to different
expectations. Water authorities believe that citizens see water resource
management as a task for authorities only, which implies that citizens do not
want to be involved. Nevertheless, most water authorities agree that they
need the observations of citizens for their work, as expressed in several
studies related to flood risk management (Wehn and Evers, 2014; Wehn et al.,
2015).</p>
      <p>This study aims to explore perspectives of government practitioners regarding
citizen science. We explore Dutch water practitioners' perceptions of citizen
motivation, acceptance in their organization, (potential) purposes and level
of citizen engagement. We address apparent knowledge gaps regarding the
motivation of professionals to embrace citizen science and specifically the
link with citizen participation. Perceptions were explored using Q
methodology. With Q methodology we identified a set of viewpoints that
describe the variation in opinions. This variation we consider meaningful for
the design and implementation of citizen science at water boards in the
Netherlands. In addition to providing insight into the viewpoints of Dutch
practitioners, this study aims to develop a methodology to study perspectives
regarding citizen science in a wider range of countries and professionals.</p>
</sec>
<sec id="Ch1.S2">
  <title>Method</title>
      <p>The study uses Q methodology (Van Exel and De Graaf, 2005; Watts and Stenner, 2012) to find viewpoints on citizen science
among employees of the Dutch water authorities. Q methodology is a relatively
uncommon method in water resource management (e.g. Raadgever et al., 2008),
but it is a popular method in social sciences fields, such as political
science and psychology (Cools et al., 2009). The strengths of the Q
methodology are that it combines qualitative and quantitative aspects and
that it is statistically robust, with small samples of 30–40 people (Van
Exel and De Graaf, 2005; Watts and Stenner, 2012). For this research Q
methodology has the advantage over fully quantitative approaches in that we
could explore a wide spectrum of viewpoints, whereas quantitative methods
would arrive easily at averaged values. Quantitative methods would likely not
reveal the distinguishing elements that are in our opinion crucial to be
aware of for effective implementation of citizen science. Q methodology has
the advantage over fully qualitative methods that it reduces the variation in
opinions to a representative small set of viewpoints in a trackable,
relatively little biased way (further discussed below).</p>
      <p>This section provides a short description of the methodology and
research-specific details. We kindly refer readers interested in more
methodological details to Van Exel and De Graaf (2005) for a quick
introduction, and Watts and Stenner (2012) or Brown (1980) for an elaboration
on the philosophy and statistical basis. In conducting this research we
closely followed the guidelines of Watts and Stenner (2012). Our Q
methodological research consisted of seven stages summarized in Fig. 1 and
elaborated below.</p>

      <?xmltex \floatpos{t}?><fig id="Ch1.F1" specific-use="star"><caption><p>Flowchart of the steps of the Q methodological research approach.
Based on Watts and Stenner (2012). For clarity reasons, we choose to describe
Stage 5 in Watts and Stenner (2012) as three separate stages.</p></caption>
        <?xmltex \igopts{width=369.885827pt}?><graphic xlink:href="https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017-f01.pdf"/>

      </fig>

      <p>The first stage aimed at collecting as wide a range as possible of opinions
on the topic of citizen science for water quality monitoring (discourse) and
documenting them in statements (concourse). We held 10 semi-structured
interviews with employees of water authorities, nature managers and citizen
organizations, from which 181 statements were derived. Additionally, we
organized a structured group discussion about citizen science with water
professionals at the afore-mentioned workshop, which resulted in an
additional 21 statements. To collect a wider range of opinions, we organized
a focus group meeting with five middle-aged woman of an informal walking club
with high potential to participate in citizen science, which resulted in an
additional 20 statements. Finally, benefits and downsides of citizen science
were extracted from the literature, which resulted in an additional seven
statements. To reduce researcher bias, we based statements as much and as
closely as possible on quotes. The final concourse (all possible statements
on a topic) consisted of 229 statements. Four themes could be identified in
the concourse: (I) citizen motivation, (II) acceptance of citizen science at
the water authority, (III) purposes of citizen science for professionals and
(IV) level of citizen engagement.</p>
      <p><?xmltex \hack{\newpage}?>In Stage 2 the concourse was reduced to the so-called Q-set set of 48
statements that still reflected the full discourse. We first reduced the
concourse to a preliminary Q-set of 65 statements by excluding or reframing
statements that were similar to others, out of scope or ambiguous. Six
masters students between the ages of 22 and 25 tested this preliminary Q-set.
Two female students had a major in water resource management; the other two
female and two male students had a different major (medicine, mathematics,
mechanical engineering and management studies). We instructed them to sort
the statements and list statements that they did not understand, found
similar in meaning or considered irrelevant, and improved the preliminary
Q-set based on their feedback. Table 1 contains the final Q-set of 46
statements; the Roman numerals indicate to what theme the statement
contributes.</p>
      <p>In Stage 3, the P-set (i.e. the group of participants) was sampled using both
a structured approach with three criteria and snowball sampling. Flood risk
was a first criterion, as water authorities with a high flood risk face
different challenges than water authorities with a lower flood risk. Age
(expressed in years since the last reform or merger with another water
authority) was a second criterion, as a recent reform suggests the
organization may be more susceptible to innovation, such as citizen science.
Location (within or outside the urban conglomerate Randstad) was a third
criterion, because interviewees in the semi-structured interviews held to
define the discourse suggested a different relationship between water
authority and population in rural and urban areas. The structured approach
resulted in eight water authorities representing a combination of these
criteria. In each water authority we approached the ecologists because we
expected them to be more familiar with the concept of citizen science and we
had access to a list of ecologists per water authority. Additionally we used
snowball sampling. We asked the participating ecologists to recruit
colleagues with a similar opinion, to enhance overlap between individual
opinions, and, with different opinions, to increase the diversity of
opinions. Also, we asked participants to recommend someone with an opposing
opinion, in order to discover as many viewpoints as possible. Participants
20, 24, 25, 30 and 31 out of 33 were recruited with this strategy, with the
aim that they would belong to different (new) viewpoints. Two to six people
with different positions were interviewed per water authority, which resulted
in interviews with 1 politician, 20 policy advisors, 10
ecologists/hydrologists and 2 field staff members.</p>
      <p>Next, in Stage 4, the Q-sorts (the actual arranging sorting process) took
place in four sub-steps, taking a total time of 60 to 75 min. First, the
first author gave three examples of consultative citizen science to all
participants, to ensure everyone had a basic level of understanding of
citizen science. These examples were
<?xmltex \hack{\newpage}?>
<list list-type="bullet"><list-item><p>the Dutch garden bird count (<uri>www.tuintelling.nl</uri>);</p></list-item><list-item><p>iSPEX: citizens measured particulate matter with a smartphone device
called iSPEX (Snik et al., 2014, p. 7351); and</p></list-item><list-item><p>water level monitoring by citizens in a Dutch water authority
(UvW, 2015b, p. 15).</p></list-item></list></p>
      <p>Second, participants pre-sorted the statements into three piles: agree,
disagree and neutral. Third, they made a final sorting of the statements in a
fixed distribution (see Fig. 2). Finally, the first author held a structured
post-sorting interview. Post-sorting interviews were included in this study,
because they can provide in-depth insight into the beliefs and values
underlying the sorts and allow for an analysis based on the participants'
rationale rather than on the available literature or the researcher's bias
(Gallagher and Porock, 2010). Discussing all statements with participants
would have been preferable, but was not feasible given the available time for
this study and the geographic spreading of participants. In a structured
interview, participants explained their reasoning for the statements in
categories <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4 and <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4 and (if time allowed) any statement of their choice.
Participant's afterthoughts were recorded, transcribed and categorized per
statement and per factor.</p>

      <?xmltex \floatpos{t}?><fig id="Ch1.F2"><caption><p>The fixed distribution used in this study. The participant places
the two statements that he agrees most with in the <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4 column, the next
three statements in the <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3 column, etc. The process is repeated for the
disagree statements.</p></caption>
        <?xmltex \igopts{width=227.622047pt}?><graphic xlink:href="https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017-f02.png"/>

      </fig>

      <p>Next, in Stage 5, a factor analysis was performed with software package
PQMethod, version 3.2.1. PQMethod is used to perform a factor analysis and is
frequently used in Q methodological research (Van Exel and De Graaf, 2005;
Cools et al., 2009; Raadgever et al., 2008; Watts and Stenner, 2012). A
factor analysis is a statistical method to describe variability in a set of
correlated variables, in this case the ranking of statements by individuals,
by a smaller number of factors. We included three factors with an eigenvalue
above 1 (recommended by Watts and Stenner, 2012) for further analysis. The
factor analysis also showed which people load on which factor, i.e. which
people have a perspective resembling that factor. People can load on none,
one or multiple factors. For this a threshold was used, the significant
factor loading (SFL). In this study we followed Watts and Stenner (2012) and
used a SFL of 0.38. Factors were optimized using factor rotation, which aims
to have as many people as possible load on a single factor rather than loading on two factors
simultaneously. The rotating process does not alter the results themselves,
but changes the researcher's observation position in order to optimize the
loading of each Q-sort on a single factor (Watts and Stenner, 2012, p. 118).
A manual rotation was preferred above the built-in Varimax rotation of the
PQMethod, because it has a lower inter-factor correlation and thus results in
more distinct viewpoints.</p>

      <?xmltex \floatpos{t}?><fig id="Ch1.F3" specific-use="star"><caption><p>A summary of the mechanisms used to reduce bias. In two columns
actions are listed to reduce researcher bias (left) and selection bias
(right).</p></caption>
        <?xmltex \igopts{width=426.791339pt}?><graphic xlink:href="https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017-f03.pdf"/>

      </fig>

      <p>Next, in Stage 6, a weighted average was used to create an illusory person
with a factor loading of 1.0 per factor, i.e. a hypothetical person who has
fully adopted this factor. Following Watts and Steiner (2012) Q-sorts with
loadings that exceed the SFL of 0.38 for a factor were incorporated to
compute the weighted average for that factor. See Table 2 for the factor
arrays and Table 3 for the final factor loadings. It must be noted that the
number of people loading on a factor cannot be used to determine the
distribution of viewpoints in the total population without additional
(quantitative) research.</p>
      <p>The final stage, Stage 7, is data analysis, where the factor arrays were
translated into a viewpoint narrative. We followed the guidelines of Watts
and Stenner (2012), Gallagher and Porock (2010) and Cools et al. (2009), and
used distinguishing statements. We created a narrative of the <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4 and <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4
ranked statements and the statements ranked highest in a single factor array,
meaning this statement is ranked lower in all other factors, and vice versa.
For example, in Factor A, Statements 2 and 9 ranked with <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4 and Statements
10 and 41 with <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4, and Statement 34 is an example of a Statement ranked
highest in factor A. Factor A ranks it <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2, compared to <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1 and <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1 in
Factors B and C. Hence, Statements 2, 9, 10, 41 and 34 were included in the
viewpoint narrative of Factor A. In addition to this mechanism of
distinguishing statements, two other mechanisms were introduced to reduce
researcher bias. First, we conducted post-sorting interviews in order to be
able to incorporate participant's underlying values and assumptions in the
process of interpretation. During these interviews we also checked whether in
their opinion all relevant aspects were covered by the Q-set. Second, we
showed all participants an initial version of the narratives and asked
whether they recognized themselves in their assigned narrative viewpoint and
why (or why not). In addition we presented the results to employees of the
Dutch water authorities on two occasions (Delfland Scriptieprijs uitreiking
21 January 2016 and STOWA Monitoringcongres 19 April 2016) to collect feedback. An overview of mechanisms used to
reduce bias in all steps of the research is given in Fig. 3.</p>

      <?xmltex \floatpos{t}?><fig id="Ch1.F4" specific-use="star"><caption><p>Distribution of interview fragments available per viewpoint and per
statement. Interview fragments of people that load to no or two viewpoints
were categorized as “neutral”.</p></caption>
        <?xmltex \igopts{width=497.923228pt}?><graphic xlink:href="https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017-f04.png"/>

      </fig>

</sec>
<sec id="Ch1.S3">
  <title>Results</title>
      <p>The 33 Q-sorts resulted in the identification of three factors from which
three viewpoints were derived: A “Citizen participation for data
application”, B “Water authority in control”, and C “Education and
sharing local knowledge”. The choice for three factors was based on the
explained variance of the first four factors before rotation that displayed
with 53, 8, 6 and 1 % a cut-off after the third factor that was supported
by qualitative information from the interviews. Table 1 and the radar charts
presented in Fig. 5 show how an individual would rank the items if that
person were representing that factor 100 %. For example, Statement 9
(“Citizen Science enables the collection of large amounts of measurements”,
Theme III Purposes of Citizen Science) would be placed under most agree
(column <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4) by a person with Factor A, under agree (column <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2) for Factor
B and under neutral (column 0) for Factor C. None of the viewpoints disagrees
with Statement 9, but the difference between Viewpoints A and C is evident.
Except for Statement 2 (“Citizen Science is important, since it contributes
to increasing water awareness”), with which all viewpoints fully agree, and
Statement 35 (“If citizens are structurally contributing, they should be
compensated for that”), differences were found between the viewpoints that
will be further discussed below. The factors provided a quite clear
separation of the participating practitioners in groups given Table 2. Out of
33 participants, 21 loaded significantly and uniquely on Factor A, 4 on
Factor B and 2 on Factor C. Three participants loaded significantly on
Factors A and C, one on Factors A and B and one on Factors B and C. One
participant did not load significantly on any of the factors.</p>
      <p>The remainder of this section contains the three viewpoint narratives. The
term viewpoint is used to refer to the factor's interpretation, for which we
made use of the quantitative factor arrays in Table 1 and qualitative quotes
from participants loading significantly on that factor (see Table 2). The
narratives are based on absolute results (agree or disagree), the relative
results (an item is ranked higher or lower in Viewpoint A than Viewpoints B
and C) and characteristic interview items. Item rankings are presented in the
following format: (item number: item ranking) such that (2: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4) means
item 02 is ranked <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4 in this viewpoint. Interview fragments are integrated
in the narratives as a quote followed by the letter Q and a number indicating
the source. For example (“quote” – Q1) means the quote comes from the
Q-sort and thus participant 1. Figure 4 shows the availability of interview
fragments per factor and per statement. Most fragments (125) were available
to interpret Factor A as most people loaded significantly to that Factor A
and less for Factors B (20 fragments) and C (10 fragments). As an intended
result of the interview technique (see Stage 4 in the Method Section), most
fragments were available for statements with particularly low or high
rankings, and there was a positive relation between the level of disagreement
among factors and the number of interview fragments. The higher the total
absolute difference between the Factors A, B and C, the higher the number of
interview fragments.</p>

<?xmltex \floatpos{p}?><table-wrap id="Ch1.T1" specific-use="star"><caption><p>Final factor arrays; the numbers in columns A, B and C are the
theoretical item score for a person whose viewpoint is 100 % that factor.
Roman numerals indicate the theme category (cat) of the item. I is citizen
motivation, II is water authority acceptance of citizen science, III is
purposes, and IV is level of engagement.</p></caption><oasis:table frame="topbot"><?xmltex \begin{scaleboxenv}{.92}[.92]?><oasis:tgroup cols="6">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="justify" colwidth="398.338583pt"/>
     <oasis:colspec colnum="3" colname="col3" align="left"/>
     <oasis:colspec colnum="4" colname="col4" align="right"/>
     <oasis:colspec colnum="5" colname="col5" align="right"/>
     <oasis:colspec colnum="6" colname="col6" align="right"/>
     <oasis:thead>
       <oasis:row rowsep="1">  
         <oasis:entry colname="col1"/>  
         <oasis:entry colname="col2">Statements (QSet)</oasis:entry>  
         <oasis:entry colname="col3">Cat</oasis:entry>  
         <oasis:entry colname="col4">A</oasis:entry>  
         <oasis:entry colname="col5">B</oasis:entry>  
         <oasis:entry colname="col6">C</oasis:entry>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>  
         <oasis:entry colname="col1">1</oasis:entry>  
         <oasis:entry colname="col2">Providing citizens with insight in water quality will only lead to unnecessary panic and questions.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">2</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science is important, since it contributes to increasing water awareness.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">4</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">4</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">4</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">3</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science is a solution to explain why you take certain measures as a water authority.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">4</oasis:entry>  
         <oasis:entry colname="col2">Water quality is an abstract concept, citizens will not understand what they measure.</oasis:entry>  
         <oasis:entry colname="col3">IV</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">5</oasis:entry>  
         <oasis:entry colname="col2">It is important to have proper communications to citizens about why values deviate from the norm and what the uncertainty  in the measured value is.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">6</oasis:entry>  
         <oasis:entry colname="col2">I would not know why citizens would not be interested in monitoring water quality.</oasis:entry>  
         <oasis:entry colname="col3">I</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">7</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science is an economical way to collect (extra) measurements.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">8</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science enables the collection of more measurements by conducting them more frequently.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">9</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science enables the collection of large amounts of measurements.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">4</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">10</oasis:entry>  
         <oasis:entry colname="col2">Measurements and observations by citizens are no valuable addition to the official monitoring network.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">11</oasis:entry>  
         <oasis:entry colname="col2">The most important goal is that the measurement data provide value to the water authority because the organization has invested its time and energy.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">12</oasis:entry>  
         <oasis:entry colname="col2">I would rather make (smart) use of existing measurements than let citizens conduct more measurements.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">13</oasis:entry>  
         <oasis:entry colname="col2">The greatest challenge is how to teach people something, if they can or want to spend little time on it.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">14</oasis:entry>  
         <oasis:entry colname="col2">Schools are especially suitable target groups to conduct these measurements, for example during a “water lesson”.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6">2</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">15</oasis:entry>  
         <oasis:entry colname="col2">The most important goal of Citizen Science is to teach people something about the environment they live in.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">16</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science is an interesting social innovation, but not suitable for actually collecting useful data.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">17</oasis:entry>  
         <oasis:entry colname="col2">Citizens' abilities are often underestimated; they are better educated and smarter than we think.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">18</oasis:entry>  
         <oasis:entry colname="col2">As a water authority we need to learn how to handle the uncertainty of alternative (cheap) measurements that originate from Citizen Science.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">19</oasis:entry>  
         <oasis:entry colname="col2">Data collection by citizens is unreliable and should not be accepted by the water authority.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">20</oasis:entry>  
         <oasis:entry colname="col2">Citizens will only participate in Citizen Science if participation is in their own interest.</oasis:entry>  
         <oasis:entry colname="col3">I</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">21</oasis:entry>  
         <oasis:entry colname="col2">Not all citizens can be trusted to conduct these measurements.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">22</oasis:entry>  
         <oasis:entry colname="col2">With a short training, citizens will be able to conduct measurements for the water authority.</oasis:entry>  
         <oasis:entry colname="col3">IV</oasis:entry>  
         <oasis:entry colname="col4">2</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6">2</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">23</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science is an interesting way to give meaning to the concept of citizen participation.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">24</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science is necessary, because it helps to decrease the awareness gap between citizens and the water authority.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">25</oasis:entry>  
         <oasis:entry colname="col2">By using Citizen Science, the water authority shows that it is keeping pace with the times.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">26</oasis:entry>  
         <oasis:entry colname="col2">An important advantage of Citizen Science is that it reduces citizen's resistance to projects.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">27</oasis:entry>  
         <oasis:entry colname="col2">One can connect with and involve another part of the audience using Citizen Science.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">28</oasis:entry>  
         <oasis:entry colname="col2">As long as Citizen Science is not included in the policy at the top levels, the water authority should not invest in it.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">29</oasis:entry>  
         <oasis:entry colname="col2">It is a major bottleneck to create support within the water authority for the deployment of Citizen Science.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">30</oasis:entry>  
         <oasis:entry colname="col2">The water authority will benefit from using Citizen Science in conducting its tasks, because fewer (financial) resources are available.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">31</oasis:entry>  
         <oasis:entry colname="col2">The conservative character of my organization is a major bottleneck for Citizen Science.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">32</oasis:entry>  
         <oasis:entry colname="col2">The organization is not equipped to work with large groups of citizen scientists.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">33</oasis:entry>  
         <oasis:entry colname="col2">My organization has no capacity to work with all this data.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">34</oasis:entry>  
         <oasis:entry colname="col2">The water authority should incorporate in its policy how to deploy and stimulate Citizen Science more.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">35</oasis:entry>  
         <oasis:entry colname="col2">If citizens are structurally contributing, they should be compensated for that.</oasis:entry>  
         <oasis:entry colname="col3">I</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6">0</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">36</oasis:entry>  
         <oasis:entry colname="col2">If citizens collect data for the water authority, they should have a say in the measures taken afterwards.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">37</oasis:entry>  
         <oasis:entry colname="col2">Citizens often have local knowledge and the water authority should use this knowledge.</oasis:entry>  
         <oasis:entry colname="col3">IV</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">4</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">4</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">38</oasis:entry>  
         <oasis:entry colname="col2">Citizen Science is important, because it gives insight into the problems that citizens are concerned with.</oasis:entry>  
         <oasis:entry colname="col3">IV</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">39</oasis:entry>  
         <oasis:entry colname="col2">Citizens should have insight in the most recent information of the water quality that is available with the water authority.</oasis:entry>  
         <oasis:entry colname="col3">IV</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">40</oasis:entry>  
         <oasis:entry colname="col2">If you provide citizens with a reference framework, they themselves can validate their data.</oasis:entry>  
         <oasis:entry colname="col3">IV</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">41</oasis:entry>  
         <oasis:entry colname="col2">I do not want citizens to interfere with our work.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">42</oasis:entry>  
         <oasis:entry colname="col2">The water authority should maintain control of conducting measurements, since the water authority is indeed responsible.</oasis:entry>  
         <oasis:entry colname="col3">IV</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">43</oasis:entry>  
         <oasis:entry colname="col2">I think the creation of Citizen Science does not fall within the tasks of the water authority.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">44</oasis:entry>  
         <oasis:entry colname="col2">I do not have a full image of what is possible with Citizen Science.</oasis:entry>  
         <oasis:entry colname="col3">II</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5">0</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">45</oasis:entry>  
         <oasis:entry colname="col2">An important caveat is that citizens will expect that their measurements will have a direct influence on policy.</oasis:entry>  
         <oasis:entry colname="col3">III</oasis:entry>  
         <oasis:entry colname="col4">0</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">46</oasis:entry>  
         <oasis:entry colname="col2">Citizens cannot be motivated to participate in such projects for a long period.</oasis:entry>  
         <oasis:entry colname="col3">I</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>  
         <oasis:entry colname="col5"><inline-formula><mml:math display="inline"><mml:mrow><mml:mo>+</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1</oasis:entry>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup><?xmltex \end{scaleboxenv}?></oasis:table></table-wrap>

      <?xmltex \floatpos{t}?><fig id="Ch1.F5" specific-use="star"><caption><p>Theoretical item score for a person whose viewpoint is 100 %
Factor A, B or C. Statements are clustered per theme (citizen motivation,
water authority acceptance, purposes and level) in four radar charts. Numbers
in brackets refer to the full statement given in Table 1.</p></caption>
        <?xmltex \igopts{width=426.791339pt}?><graphic xlink:href="https://hess.copernicus.org/articles/21/153/2017/hess-21-153-2017-f05.png"/>

      </fig>

<?xmltex \floatpos{t}?><table-wrap id="Ch1.T2" specific-use="star"><caption><p>Final factor loadings after rotation. Sorts in bold indicate that a
person's loading exceeds the significant factor loading (SFL) and a person's
viewpoint thus resembles Viewpoint A, B or C. Also included are
characteristics of respondents: water authority (Identification (ID),
elevation above sea level (El), located in Randstad (RS) and recently
reformed (since 2005)) and individual characteristics (sex, age and function
group).</p></caption><oasis:table frame="topbot"><oasis:tgroup cols="11">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="right"/>
     <oasis:colspec colnum="3" colname="col3" align="right"/>
     <oasis:colspec colnum="4" colname="col4" align="right" colsep="1"/>
     <oasis:colspec colnum="5" colname="col5" align="left"/>
     <oasis:colspec colnum="6" colname="col6" align="center"/>
     <oasis:colspec colnum="7" colname="col7" align="left"/>
     <oasis:colspec colnum="8" colname="col8" align="left" colsep="1"/>
     <oasis:colspec colnum="9" colname="col9" align="left"/>
     <oasis:colspec colnum="10" colname="col10" align="center"/>
     <oasis:colspec colnum="11" colname="col11" align="left"/>
     <oasis:thead>
       <oasis:row>  
         <oasis:entry colname="col1"><bold/></oasis:entry>  
         <oasis:entry rowsep="1" namest="col2" nameend="col4" align="center" colsep="1">Factors </oasis:entry>  
         <oasis:entry rowsep="1" namest="col5" nameend="col8" align="center" colsep="1">Water authority </oasis:entry>  
         <oasis:entry rowsep="1" namest="col9" nameend="col11" align="center">Individual respondent </oasis:entry>
       </oasis:row>
       <oasis:row rowsep="1">  
         <oasis:entry colname="col1">Sort</oasis:entry>  
         <oasis:entry colname="col2">A</oasis:entry>  
         <oasis:entry colname="col3">B</oasis:entry>  
         <oasis:entry colname="col4">C</oasis:entry>  
         <oasis:entry colname="col5">ID</oasis:entry>  
         <oasis:entry colname="col6">El.</oasis:entry>  
         <oasis:entry colname="col7">RS</oasis:entry>  
         <oasis:entry colname="col8">RR</oasis:entry>  
         <oasis:entry colname="col9">Sex</oasis:entry>  
         <oasis:entry colname="col10">Age</oasis:entry>  
         <oasis:entry colname="col11">Function group</oasis:entry>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>1</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.74</bold></oasis:entry>  
         <oasis:entry colname="col3">0.18</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.01</oasis:entry>  
         <oasis:entry colname="col5">A</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Politician</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>2</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.49</bold></oasis:entry>  
         <oasis:entry colname="col3">0.18</oasis:entry>  
         <oasis:entry colname="col4">0.35</oasis:entry>  
         <oasis:entry colname="col5">A</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>3</bold></oasis:entry>  
         <oasis:entry colname="col2">0.35</oasis:entry>  
         <oasis:entry colname="col3"><bold>0.55</bold></oasis:entry>  
         <oasis:entry colname="col4">0.18</oasis:entry>  
         <oasis:entry colname="col5">A</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>4</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.49</bold></oasis:entry>  
         <oasis:entry colname="col3">0.36</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.03</oasis:entry>  
         <oasis:entry colname="col5">B</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20-45</oasis:entry>  
         <oasis:entry colname="col11">Advisor (water quality/ecology)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>5</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.69</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.07</oasis:entry>  
         <oasis:entry colname="col4">0,29</oasis:entry>  
         <oasis:entry colname="col5">C</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20-45</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>6</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.71</bold></oasis:entry>  
         <oasis:entry colname="col3">0.09</oasis:entry>  
         <oasis:entry colname="col4">0.32</oasis:entry>  
         <oasis:entry colname="col5">C</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>7</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.45</bold></oasis:entry>  
         <oasis:entry colname="col3"><bold>0.62</bold></oasis:entry>  
         <oasis:entry colname="col4">0.12</oasis:entry>  
         <oasis:entry colname="col5">D</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Field staff</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>8</bold></oasis:entry>  
         <oasis:entry colname="col2">0.31</oasis:entry>  
         <oasis:entry colname="col3"><bold>0.46</bold></oasis:entry>  
         <oasis:entry colname="col4"><bold>0.43</bold></oasis:entry>  
         <oasis:entry colname="col5">D</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Advisor (ecological monitoring)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>9</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.79</bold></oasis:entry>  
         <oasis:entry colname="col3">0.19</oasis:entry>  
         <oasis:entry colname="col4">0.06</oasis:entry>  
         <oasis:entry colname="col5">D</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>10</bold></oasis:entry>  
         <oasis:entry colname="col2"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.05</oasis:entry>  
         <oasis:entry colname="col3">0.22</oasis:entry>  
         <oasis:entry colname="col4"><bold>0.66</bold></oasis:entry>  
         <oasis:entry colname="col5">D</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Advisor (ecology)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>11</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.64</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.17</oasis:entry>  
         <oasis:entry colname="col4">0.03</oasis:entry>  
         <oasis:entry colname="col5">C</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>12</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.72</bold></oasis:entry>  
         <oasis:entry colname="col3">0,12</oasis:entry>  
         <oasis:entry colname="col4"><bold>0.49</bold></oasis:entry>  
         <oasis:entry colname="col5">C</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>13</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.75</bold></oasis:entry>  
         <oasis:entry colname="col3">0.01</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.12</oasis:entry>  
         <oasis:entry colname="col5">C</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Advisor (innovation)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>14</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.69</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.06</oasis:entry>  
         <oasis:entry colname="col4"><bold>0.41</bold></oasis:entry>  
         <oasis:entry colname="col5">C</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Field staff</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>15</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.84</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.06</oasis:entry>  
         <oasis:entry colname="col4">0.30</oasis:entry>  
         <oasis:entry colname="col5">E</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>16</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.64</bold></oasis:entry>  
         <oasis:entry colname="col3">0.33</oasis:entry>  
         <oasis:entry colname="col4"><bold>0.42</bold></oasis:entry>  
         <oasis:entry colname="col5">E</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Advisor (permits and assessment)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>17</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.58</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.05</oasis:entry>  
         <oasis:entry colname="col4">0.25</oasis:entry>  
         <oasis:entry colname="col5">E</oasis:entry>  
         <oasis:entry colname="col6"><inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Advisor (water system)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>18</bold></oasis:entry>  
         <oasis:entry colname="col2"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.04</oasis:entry>  
         <oasis:entry colname="col3"><bold>0.55</bold></oasis:entry>  
         <oasis:entry colname="col4">0.24</oasis:entry>  
         <oasis:entry colname="col5">F</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>19</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.69</bold></oasis:entry>  
         <oasis:entry colname="col3">0.01</oasis:entry>  
         <oasis:entry colname="col4">0.17</oasis:entry>  
         <oasis:entry colname="col5">F</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Hydrologist</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>20</bold></oasis:entry>  
         <oasis:entry colname="col2">0.32</oasis:entry>  
         <oasis:entry colname="col3"><bold>0.43</bold></oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.13</oasis:entry>  
         <oasis:entry colname="col5">F</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>21</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.73</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.09</oasis:entry>  
         <oasis:entry colname="col4">0.32</oasis:entry>  
         <oasis:entry colname="col5">F</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>22</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.59</bold></oasis:entry>  
         <oasis:entry colname="col3">0.25</oasis:entry>  
         <oasis:entry colname="col4">0.04</oasis:entry>  
         <oasis:entry colname="col5">F</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">No</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Hydrologist</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>23</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.82</bold></oasis:entry>  
         <oasis:entry colname="col3">0.17</oasis:entry>  
         <oasis:entry colname="col4">0.01</oasis:entry>  
         <oasis:entry colname="col5">G</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Advisor (innovation)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>24</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.74</bold></oasis:entry>  
         <oasis:entry colname="col3">0.04</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.12</oasis:entry>  
         <oasis:entry colname="col5">G</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Advisor (innovation)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>25</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.45</bold></oasis:entry>  
         <oasis:entry colname="col3">0.00</oasis:entry>  
         <oasis:entry colname="col4">0.23</oasis:entry>  
         <oasis:entry colname="col5">G</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Ecologist</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>26</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.80</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.06</oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.07</oasis:entry>  
         <oasis:entry colname="col5">G</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Hydrologist</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>27</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.85</bold></oasis:entry>  
         <oasis:entry colname="col3">0.15</oasis:entry>  
         <oasis:entry colname="col4">0.23</oasis:entry>  
         <oasis:entry colname="col5">G</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>28</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.54</bold></oasis:entry>  
         <oasis:entry colname="col3">0.20</oasis:entry>  
         <oasis:entry colname="col4">0.27</oasis:entry>  
         <oasis:entry colname="col5">H</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>29</bold></oasis:entry>  
         <oasis:entry colname="col2">0.34</oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.03</oasis:entry>  
         <oasis:entry colname="col4">0.34</oasis:entry>  
         <oasis:entry colname="col5">H</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Project leader</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>30</bold></oasis:entry>  
         <oasis:entry colname="col2">0.30</oasis:entry>  
         <oasis:entry colname="col3"><bold>0.39</bold></oasis:entry>  
         <oasis:entry colname="col4"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.09</oasis:entry>  
         <oasis:entry colname="col5">A</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">Yes</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>31</bold></oasis:entry>  
         <oasis:entry colname="col2">0.27</oasis:entry>  
         <oasis:entry colname="col3">0.23</oasis:entry>  
         <oasis:entry colname="col4"><bold>0.43</bold></oasis:entry>  
         <oasis:entry colname="col5">B</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">M</oasis:entry>  
         <oasis:entry colname="col10">45–65</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>32</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.77</bold></oasis:entry>  
         <oasis:entry colname="col3">0.21</oasis:entry>  
         <oasis:entry colname="col4">0.10</oasis:entry>  
         <oasis:entry colname="col5">B</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Advisor (water quality)</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1"><bold>33</bold></oasis:entry>  
         <oasis:entry colname="col2"><bold>0.79</bold></oasis:entry>  
         <oasis:entry colname="col3"><inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>0.22</oasis:entry>  
         <oasis:entry colname="col4">0.34</oasis:entry>  
         <oasis:entry colname="col5">B</oasis:entry>  
         <oasis:entry colname="col6">–</oasis:entry>  
         <oasis:entry colname="col7">Yes</oasis:entry>  
         <oasis:entry colname="col8">No</oasis:entry>  
         <oasis:entry colname="col9">F</oasis:entry>  
         <oasis:entry colname="col10">20–45</oasis:entry>  
         <oasis:entry colname="col11">Policy advisor</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">EV<inline-formula><mml:math display="inline"><mml:msup><mml:mi/><mml:mn mathvariant="normal">1</mml:mn></mml:msup></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col2">12.55</oasis:entry>  
         <oasis:entry colname="col3">2.29</oasis:entry>  
         <oasis:entry colname="col4">2.58</oasis:entry>  
         <oasis:entry colname="col5"/>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>  
         <oasis:entry colname="col8"/>  
         <oasis:entry colname="col9"/>  
         <oasis:entry colname="col10"/>  
         <oasis:entry colname="col11"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Var<inline-formula><mml:math display="inline"><mml:msup><mml:mi/><mml:mn mathvariant="normal">2</mml:mn></mml:msup></mml:math></inline-formula></oasis:entry>  
         <oasis:entry colname="col2">38 %</oasis:entry>  
         <oasis:entry colname="col3">7 %</oasis:entry>  
         <oasis:entry colname="col4">8 %</oasis:entry>  
         <oasis:entry colname="col5"/>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>  
         <oasis:entry colname="col8"/>  
         <oasis:entry colname="col9"/>  
         <oasis:entry colname="col10"/>  
         <oasis:entry colname="col11"/>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup></oasis:table></table-wrap>

<sec id="Ch1.S3.SS1">
  <title>Viewpoint A: “Citizen participation for data application”</title>
      <p>The people loading on Factor A (see Table 2) are a mix of hydrologists,
advisors, policy advisors, field staff and a politician. In this group are 14
men and 11 women. Eleven people are middle aged (between 45 and 65). They
represent all eight incorporated water authorities, which are located within
and outside the Randstad and have a mixture of higher and lower flood risk.
Nineteen people work at a water authority that has recently gone through an
organizational reform.</p>
      <p>People with Viewpoint A think that citizen science is important for water
authorities to increase water awareness (2: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4), because citizens are
unacquainted with the work of the water authority: “People often do not know
what the water authority is doing exactly and we do not really stand out.
Citizens sometimes really wonder what they pay tax for […]” (Q5) and
“what they can do themselves to improve water quality” (Q27). They believe
that their water authority should actively incorporate citizen science in its
policy (34: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2) and that the water authority should not wait to invest in
citizen science until it is included in top-level policies (28: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3).</p>
      <p>Additionally, people with this viewpoint value citizen science for the
collection of large amounts of data (9: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4) and for conducting measurements
more frequently (8: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3). “This data, they are an opportunity to have an
area covering insight in dynamics of water quality and ecology” – Q26. They
think that citizens can be trusted to conduct these measurements (21: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1).
Although citizen science is a social innovation and the acquired data are
less accurate, it should be accepted by the water authority (19: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3) (“I
mainly disagree strongly with the latter part of this statement” – Q13
[(…) and should not be accepted by the water authority]) and will be a
valuable addition to the official monitoring network (10: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4). These people
do not prefer the smart use of existing data to citizen science data (12:
<inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1). The organization is expected to have sufficient capacity to analyse
all the data (33: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2) at the moment, but the water authority has to learn
how to handle the uncertainty of these alternative (often more economical)
measurements (18: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2). They do not believe that the water authority needs
to maintain control of monitoring, even though water authorities are in the
end responsible for monitoring (42: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2). “This is nonsense, because a lot
is already measured by other parties” – Q25. They believe that citizens, if
provided with a reference framework, can validate their own data (40: 0) “If
they know what to do with it [the results], they can translate it to their
environment” – Q11.</p>
      <p>People with Viewpoint A think of citizen science as an interesting way to
give meaning to the concept of citizen participation (23: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3) and decrease
the gap between the water authority and citizens (24: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2). They are least
(compared to Viewpoints B and C) afraid that citizens will expect their
contribution to have a direct impact on policy (45: 0) and they do not think
citizens should get this influence (36: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2). “You should prevent that,
because manipulation [of results] is evident” – Q9. They consider citizen
science to be a solution when it comes to explaining why you undertake
certain measures (3: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1). This group further feels that citizen science
will show that the water authority is keeping pace with the times (25: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1),
although it is not a priority. Giving citizens' insight into water quality
will not lead to unnecessary questions and panic (1: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3). “Those questions
will come, but you should not be afraid, not afraid to say that you do not
know everything” – Q21. They do not fear citizen interference with their
work (41: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4).</p>
</sec>
<sec id="Ch1.S3.SS2">
  <title>Viewpoint B: “The water authority in control”</title>
      <p>The six people loading significantly on Factor B (see Table 2) form a mixture
of advisors, policy advisors and field staff. Five out of six are male and
four of them are middle aged. They work at three different water authorities;
two people work outside the Randstad. Four people work at water authorities
that have recently (after 2005) gone through an organizational reform. All
work in an area with a high flood risk.</p>
      <p>People with Viewpoint B consider citizen science important for increasing
water awareness (2: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4). In contrast to Viewpoint A, they think that the
water authority should not incorporate citizen science as part of its policy
(34: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1). They feel they do not have a full idea (yet) of what is possible
with citizen science (44: 0). However, they do think the water authority
should invest in citizen science, even if it is not yet included in top level
policies (28: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3). They do fear that citizens cannot be motivated for
long-term participation (46: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2) and will not participate unless
participation is in their own interest (20: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2). This group is more
concerned than the other two groups that the creation of a support base
within the organization will be a bottleneck (29: 0), and they are convinced
that their organization is not (yet) equipped to work with large groups of
citizen scientists (32: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3).</p>
      <p>People with Viewpoint B believe that local knowledge will be valuable for the
water authority (37: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4), as the citizen “knows his own environment better
than we do, on the small scale. We only have the broad overview in a large
area” – Q20. They strongly believe that the water authority needs to
maintain control of monitoring, because it has the final responsibility (42:
<inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3). “I know what should be done with the data in the end. If we leave it
to volunteers in this case, you have no reassurance on what comes when” –
Q3. Citizen science allows for the collection of more measurements (9: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2)
by conducting them more frequently (8: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3). They think citizens will be
able to conduct measurements after they receive a short training (22: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1),
but they do not expect that citizens will be able to validate their own data
if provided with a reference framework (40: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3). “If [data] quality is
important to you, I am not sure whether citizens can do this” – Q18. They
further question whether all citizens can be trusted in doing measurements
(21: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1). “If the citizen does not have personal interest, you have to
wait to see what happens. Then he will think: I do not feel like it, I do not
have time” – Q20. This reflects their belief that most citizens would not
be interested in participating (6: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2). This group does not believe the
water authority needs citizen science to help fulfil its tasks to compensate
for fewer financial resources (30: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1).</p>
      <p>People with this viewpoint are least convinced that citizen science will
involve another part of the public (27: 0) or that it is an interesting way
to give meaning to citizen participation (23: 1). Moreover, they believe that
citizen science should not be used to decrease the gap between citizens and
water authorities (24: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3) or to show that the water authority is keeping
pace with the times (25: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1). “If this is your reason, I think it is
rather cheap” – Q18. If citizens start collecting data for the water
authority, this group strongly feels that they should not be given more
influence over measures (36: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4), but they do fear that citizens will think
that their work will influence policy directly (45: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2). “Citizens, I
would almost say per definition, cannot do that [balance interests], they
just want to do what they want” – Q30. People in this group do not fear
questions or panic from citizens (1: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4). “If you are so suspicious
towards your citizens, you have to question your role as government” – Q3.</p>
</sec>
<sec id="Ch1.S3.SS3">
  <title>Viewpoint C: “Education and sharing local knowledge”</title>
      <p>The six people significantly loading on Factor C (see Table 2) are a mix of
advisors, policy advisors and field staff. Four people are middle aged and
three of them are male. They work at four different water authorities, three
respondents within and three outside the Randstad. Three people work at a
water authority that has recently (after 2005) gone through an organizational
reform. Three out of six people with Viewpoint C work in an area with a lower
flood risk.</p>
      <p>People with Viewpoint C think that citizen science is important, because it
contributes to increasing water awareness (2: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4). The most important
purpose for this group is to teach people something about their environment
(15: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3), and especially schools are considered to be a good target
audience (14: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2). “It is a good way to keep them [students] engaged” –
Q10. They think that citizens will understand what they measure, even though
water quality is an abstract concept (4: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3). “I think this is an
offensive comment toward the citizens, as if they are stupid” – Q10. They
believe it is possible to teach people something within a short period of
time (13: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>1) and they find it difficult to think of reasons why people
would not be interested in water quality (6: 0) compared to the other two
groups. In contrast to Viewpoint B, they do think that citizens will
participate, even if participation does not directly serve their own
interests (20: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2). “I participate as a citizen in a sort of science
project, I do not do that for my own benefit, but because I like it and want
to contribute. I think I am not the only one” – Q10. Also, the conservative
character of water authorities is not considered a major bottleneck (31:
<inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>2).</p>
      <p>They feel that the water authority should use the local knowledge that
citizens have (37: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>4), but they consider citizen science to be merely a
social innovation, rather than a way to collect useful data, compared to the
other viewpoints (16: 0). This is reflected in their relatively small support
of the idea that citizen science will allow for collecting large amounts of
data (9: 0) and for conducting measurements more frequently (8: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1). “It
is mainly supportive material and not a replacement of existing sources,
because it is invalidated and uncertified information. I do not think that
will fit” – Q31. They strongly reject the view that citizens should not
interfere with their work (41: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4), although they believe the water
authority should stay in control (42: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2). “In my opinion information is
essential for policy to be good, [so] I think they should be collected by a
professional” – Q10. People in this group believe that citizens will not be
able to validate their own data (40: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>3).</p>
      <p>People with Viewpoint C consider citizen science to be a good way to bind and
involve another part of the audience (27: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>3), to decrease the gap between
citizens and water authorities (24: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2) and, to a lesser extent, to reduce
citizens' resistance to projects (26: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1). A caveat could be that citizens
will expect their measurements to have a direct influence on policy (45:
<inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>1), even though they should not be given a say in the measures taken
afterwards (36: <inline-formula><mml:math display="inline"><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula>). “For me these are two separated tracks. […] they
have this influence via the representatives that they can elect for the
board” – Q31. Citizens should be given insight into the most recent
information about water quality that is available with the water authority
(39: <inline-formula><mml:math display="inline"><mml:mo>+</mml:mo></mml:math></inline-formula>2). “I believe that citizens and everyone have the right to get
information from us” – Q10. These people strongly disagree that providing
citizens with insight into water quality will lead to unnecessary panic and
questions (1: <inline-formula><mml:math display="inline"><mml:mo>-</mml:mo></mml:math></inline-formula>4).</p>
</sec>
</sec>
<sec id="Ch1.S4">
  <title>Discussion</title>
      <p>This study identified three different viewpoints with regard to citizen
science derived with Q methodology. Participants sorted statements about four
themes described in the Introduction and Method sections. In this discussion
we first (Sect. 4.1) reflect on these four themes and relate the results to
the literature, and second (Sect. 4.2) discuss the limitations of the
research and make recommendations for future research.</p>
<sec id="Ch1.S4.SS1">
  <title>Reflection on the four themes</title>
      <p>The statements contributed to one of the four themes: (I) citizen motivation,
(II) acceptance of citizen science at the water authority, (III) purposes and
(IV) level of citizen participation. Several statements related to perceived
citizen motivation by the participants (Theme I in Table 1 and the upper-left
radar chart in Fig. 5) and acceptance of citizen science at the water
authority (Theme II and the upper-right radar chart in Fig. 5). Trust in
citizens' motivations and commitment ranged from low in Viewpoint B to high
in Viewpoint C. The assessment of high, low or medium was based on the
attitude that emerged from the ranking of statements in Themes I and II.
Viewpoint C clearly had the most positive attitude towards citizens'
motivations and B the least positive. For Theme II the distinction was less
clear, although all viewpoints express a rather positive attitude. All three
viewpoints are rather positive about implementing citizen science (perhaps
due to some volunteer bias discussed in Sect. 4.2), although Viewpoints A and
B are concerned with, respectively, the image of the water authority, and
with the organizational capacity and a lack of internal support. Table 3
summarizes these findings. Table 4 summarizes the results with regard to the
support per viewpoint for the purposes listed in the Introduction (Theme
III). The same has been done for the roles that citizens can have (Theme IV)
in Table 5 according to the classification of Bonney et al. (2009).</p>

<?xmltex \floatpos{t}?><table-wrap id="Ch1.T3"><caption><p>Summary of the perception of the level of citizen motivations to
contribute (Theme 1) and the level of acceptance of citizen science at the
water authority (Theme II) for Viewpoints A, B and C.</p></caption><oasis:table frame="topbot"><oasis:tgroup cols="4">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="left"/>
     <oasis:colspec colnum="3" colname="col3" align="left"/>
     <oasis:colspec colnum="4" colname="col4" align="left"/>
     <oasis:thead>
       <oasis:row rowsep="1">  
         <oasis:entry colname="col1"/>  
         <oasis:entry colname="col2">A</oasis:entry>  
         <oasis:entry colname="col3">B</oasis:entry>  
         <oasis:entry colname="col4">C</oasis:entry>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>  
         <oasis:entry colname="col1">Theme 1: Citizen motivations</oasis:entry>  
         <oasis:entry colname="col2">Medium</oasis:entry>  
         <oasis:entry colname="col3">Low</oasis:entry>  
         <oasis:entry colname="col4">High</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">to contribute</oasis:entry>  
         <oasis:entry colname="col2"/>  
         <oasis:entry colname="col3"/>  
         <oasis:entry colname="col4"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Theme 2: Acceptance of citizen</oasis:entry>  
         <oasis:entry colname="col2">High</oasis:entry>  
         <oasis:entry colname="col3">Medium</oasis:entry>  
         <oasis:entry colname="col4">High</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">science at the water authority</oasis:entry>  
         <oasis:entry colname="col2"/>  
         <oasis:entry colname="col3"/>  
         <oasis:entry colname="col4"/>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup></oasis:table></table-wrap>

<?xmltex \floatpos{t}?><table-wrap id="Ch1.T4"><caption><p>Overview of Theme III (purposes for citizen science) as supported in
Viewpoints A, B and C. Purposes clearly indicated are marked with an X. (x)
indicates that support for this purpose is not convincing.</p></caption><oasis:table frame="topbot"><oasis:tgroup cols="4">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="center"/>
     <oasis:colspec colnum="3" colname="col3" align="center"/>
     <oasis:colspec colnum="4" colname="col4" align="center"/>
     <oasis:thead>
       <oasis:row>  
         <oasis:entry colname="col1"/>  
         <oasis:entry colname="col2">Viewpoint</oasis:entry>  
         <oasis:entry colname="col3">Viewpoint</oasis:entry>  
         <oasis:entry colname="col4">Viewpoint</oasis:entry>
       </oasis:row>
       <oasis:row rowsep="1">  
         <oasis:entry colname="col1"/>  
         <oasis:entry colname="col2">A</oasis:entry>  
         <oasis:entry colname="col3">B</oasis:entry>  
         <oasis:entry colname="col4">C</oasis:entry>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>  
         <oasis:entry colname="col1">Increase knowledge</oasis:entry>  
         <oasis:entry colname="col2">X</oasis:entry>  
         <oasis:entry colname="col3">X</oasis:entry>  
         <oasis:entry colname="col4">X</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Improve methods</oasis:entry>  
         <oasis:entry colname="col2">X</oasis:entry>  
         <oasis:entry colname="col3"/>  
         <oasis:entry colname="col4"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Raise awareness</oasis:entry>  
         <oasis:entry colname="col2">X</oasis:entry>  
         <oasis:entry colname="col3">X</oasis:entry>  
         <oasis:entry colname="col4">X</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Improve management</oasis:entry>  
         <oasis:entry colname="col2">X</oasis:entry>  
         <oasis:entry colname="col3">(x)</oasis:entry>  
         <oasis:entry colname="col4"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Public education</oasis:entry>  
         <oasis:entry colname="col2"/>  
         <oasis:entry colname="col3"/>  
         <oasis:entry colname="col4">X</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Policy development</oasis:entry>  
         <oasis:entry colname="col2">X</oasis:entry>  
         <oasis:entry colname="col3"/>  
         <oasis:entry colname="col4">(x)</oasis:entry>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup></oasis:table></table-wrap>

      <p>All viewpoints embrace data collection by citizens, i.e. contributory
projects, and none support co-created projects (Table 5). Viewpoint A is
optimistic towards citizen participation in the analysis of the data (see
Statement 40), suggesting a potential for collaborative projects. Viewpoints
B and C are wary of involving citizens in these steps of the research
process. None of the viewpoints supports statements related to co-created
projects. The following explanations are illustrative of the reluctance of
participants to involve citizens in topic selection for monitoring
(Statement 38): “There can be [topics] which we think they are important,
while citizens do not find it important in the end” (Q24, Viewpoint A) and
“In that case you should answer all [these questions of citizens] and I
think our organisation is not equipped at the moment” (Q18, Viewpoint B).
Regarding the translation of results to action (Statement 36), participants
said “They [citizens] can only focus on the problems in their direct
environment, but not on the implications for a wider area” (Q19, Viewpoint
A) and “I would not go that far” (Q18, Viewpoint B). Participants mentioned
external causes such as the legal obligations a water authority has regarding
the use of standardized methods and reports for water quality monitoring.
This is consistent with previous conclusions about the responsibility of the
water authority in relation to acceptance of citizen science (Wehn and Evers,
2014). Other participants mentioned internal causes, such as the existing
procedures for citizens to influence decision-making, including complaint
procedures and the water authority general elections once every 4 years.</p>

<?xmltex \floatpos{t}?><table-wrap id="Ch1.T5" specific-use="star"><caption><p>Overview of theme IV (level of citizen participation), based
on Bonney et al. (2009). Supported activities are marked with an X. An (x)
indicates the activities are sometimes assigned to citizens.</p></caption><oasis:table frame="topbot"><?xmltex \begin{scaleboxenv}{.96}[.96]?><oasis:tgroup cols="7">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="center"/>
     <oasis:colspec colnum="3" colname="col3" align="center"/>
     <oasis:colspec colnum="4" colname="col4" align="center"/>
     <oasis:colspec colnum="5" colname="col5" align="center"/>
     <oasis:colspec colnum="6" colname="col6" align="center"/>
     <oasis:colspec colnum="7" colname="col7" align="center"/>
     <oasis:thead>
       <oasis:row>  
         <oasis:entry colname="col1"/>  
         <oasis:entry colname="col2">Contributory</oasis:entry>  
         <oasis:entry colname="col3">Collaborative</oasis:entry>  
         <oasis:entry colname="col4">Co-created</oasis:entry>  
         <oasis:entry colname="col5">Viewpoint</oasis:entry>  
         <oasis:entry colname="col6">Viewpoint</oasis:entry>  
         <oasis:entry colname="col7">Viewpoint</oasis:entry>
       </oasis:row>
       <oasis:row rowsep="1">  
         <oasis:entry colname="col1"/>  
         <oasis:entry colname="col2">projects</oasis:entry>  
         <oasis:entry colname="col3">projects</oasis:entry>  
         <oasis:entry colname="col4">projects</oasis:entry>  
         <oasis:entry colname="col5">A</oasis:entry>  
         <oasis:entry colname="col6">B</oasis:entry>  
         <oasis:entry colname="col7">C</oasis:entry>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>  
         <oasis:entry colname="col1">Choose or define question(s) for study</oasis:entry>  
         <oasis:entry colname="col2"/>  
         <oasis:entry colname="col3"/>  
         <oasis:entry colname="col4">X</oasis:entry>  
         <oasis:entry colname="col5"/>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Design data collection methodologies</oasis:entry>  
         <oasis:entry colname="col2"/>  
         <oasis:entry colname="col3">(x)</oasis:entry>  
         <oasis:entry colname="col4">X</oasis:entry>  
         <oasis:entry colname="col5"/>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Collect samples and/or record data</oasis:entry>  
         <oasis:entry colname="col2">X</oasis:entry>  
         <oasis:entry colname="col3">X</oasis:entry>  
         <oasis:entry colname="col4">X</oasis:entry>  
         <oasis:entry colname="col5">X</oasis:entry>  
         <oasis:entry colname="col6">X</oasis:entry>  
         <oasis:entry colname="col7">X</oasis:entry>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Analyse samples and data</oasis:entry>  
         <oasis:entry colname="col2">(x)</oasis:entry>  
         <oasis:entry colname="col3">X</oasis:entry>  
         <oasis:entry colname="col4">X</oasis:entry>  
         <oasis:entry colname="col5">X</oasis:entry>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Interpret data and raw conclusions</oasis:entry>  
         <oasis:entry colname="col2"/>  
         <oasis:entry colname="col3">(x)</oasis:entry>  
         <oasis:entry colname="col4">X</oasis:entry>  
         <oasis:entry colname="col5"/>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Disseminate conclusions/translate results into action</oasis:entry>  
         <oasis:entry colname="col2">(x)</oasis:entry>  
         <oasis:entry colname="col3">(x)</oasis:entry>  
         <oasis:entry colname="col4">X</oasis:entry>  
         <oasis:entry colname="col5"/>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>
       </oasis:row>
       <oasis:row>  
         <oasis:entry colname="col1">Discuss results and ask new questions</oasis:entry>  
         <oasis:entry colname="col2"/>  
         <oasis:entry colname="col3"/>  
         <oasis:entry colname="col4">X</oasis:entry>  
         <oasis:entry colname="col5"/>  
         <oasis:entry colname="col6"/>  
         <oasis:entry colname="col7"/>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup><?xmltex \end{scaleboxenv}?></oasis:table></table-wrap>

      <p>There appears to be a mismatch between the intentions of the participants
with respect to the purposes that citizen science should support and the
expressed level of trust in citizens as reflected by the envisioned
participation level. Especially people with Viewpoint C believe citizens have
noble motivations, and they trust the citizens to a great extent. This trust
is not reflected in the roles they envision for citizens, which is limited to
data collection. The same goes for Viewpoint A. People with this viewpoint
believe citizen science can serve many purposes (see Table 4). However, the
envisioned role for citizens is limited to data collection and analysis. A
lack of trust in citizens, few intentions to use the citizen scientists' data
or a lack of support for higher levels of participation might conflict with
citizens' motivations as described in the Introduction. A relation of mutual
trust is required as the basis for effective citizen science projects and
prolonged contributions by citizens (Rotman et al., 2012). Viewpoint B
reveals distrust in the commitment of citizens, citizens' intentions to
participate and their capacity (see Statements 20, 21, 22, 40 and 45).
Another important motivation for citizens is the provision of feedback on how
the data are used (e.g. Bonney et al., 2009; Rotman et al., 2012; Roy et al.,
2012), which is particularly in contrast to Viewpoint C. Viewpoint C focuses
on the goals of education (see Statements 14 and 15), with little emphasis on
the actual use of the data (see Statements 8 and 9). Concerns regarding data
quality may linger if recent development in systematic modelling and data
assimilation approaches (e.g. Clark et al., 2015; Shoaib et al., 2016; Hut et
al., 2015) would be adopted at the water authorities in the Netherlands. Such
frameworks would allow systematic tracing of the propagation of uncertainty
of citizen data in decision support tools and may identify opportunities for
citizen science in model structure determination, which can be the largest
source of uncertainty (Shoaib et al., 2016). Ottinger (2010) stressed the
need for standardized methods in citizen science and legal embedding to
increase the acceptance of the data and actual use of citizen science in
policy-making. In the Netherlands systematic data handling and
standardization may pave the way for using citizen science for public
participation, in line with the vision of the Dutch water authorities
governing body (UvW, 2015b).</p>
</sec>
<sec id="Ch1.S4.SS2">
  <title>Research limitations and recommendations</title>
      <p>Q methodology is an abductive research approach (Watts and Stenner, 2012),
which means that we tried to understand and explain the data rather than
describe it or test a hypothesis. This approach is subjective in nature.
Researcher bias was reduced where possible as presented in the framework in
Fig. 3. By collecting statements from various sources, reasonable saturation
in the statements that formed the discourse was achieved and confirmed by the
fact that no clearly new statements arose from the post-sorting interviews.
The second sampling strategy (see Stage 4 in the Method section) recruited
five participants that were expected to have different viewpoints. Two of
them had Viewpoint B and one had Viewpoint C, thus broadening the scope of
viewpoints. Still, the voluntary nature of the participation might have
attracted participants with a positive attitude towards citizen science. On
several occasions the research was presented to water authority employees.
They were surprised that all viewpoints were relatively positive about
citizen science, because they would have believed to have colleagues who were
indeed more sceptical than these viewpoints suggest.</p>
      <p>To further reduce researcher bias, we asked participants whether they
recognized themselves in the assigned viewpoint
described in an early draft of the text presented in the Results section.
Fifteen participants responded, 13 of whom fully identified with their
viewpoint, because they agreed with the viewpoints' main assertions. Two
respondents were in doubt, due to overlap between Viewpoints A and C. The
correlation between Factors A and C was 0.43, which indeed indicates that
they are interrelated and overlap. Typically, correlations above 0.39 are
considered significant (Watts and Stenner, 2012). Still, Factor A and Factor
C are considered sufficiently different to regard them as the basis for
separate viewpoints, particularly given the (in our opinion crucially)
different opinion on involving citizens in data collection. Correlations
between Factors A and B and Factors B and C were lower and not significant:
0.26 and 0.35, respectively. A mechanism to further reduce bias, beyond the
mechanisms in Fig. 3, could be to execute stages such as statement
definition, interviews and transcript analysis by independently working
researchers. However, this was not feasible within the scope of this
research.</p>
      <p>The importance of post-sorting interviews, as stated by Gallagher and
Porock (2010), was recognized in this work. In particular, statements with
multiple fragments per viewpoint illustrated this. For example, the interview
fragments for Statement 36 revealed a difference in the reasoning to exclude
citizens from decision-making on measures to realize policy goals.
Participants with Viewpoint A feared manipulation of results, while
participants with Viewpoint B emphasized the responsibility of the water
authority to take an informed decision and balance conflicting interests.
Hence, reformulation and/or splitting of Statement 36 may be considered if
the set of statements was adopted for future research. Four participants with
Viewpoint A literally recalled citizens' unfamiliarity with the tasks of the
water authorities in response to Statement 2. Time limitations of the
interviews resulted in an unequal distribution of interview fragments over
statements and viewpoints (see Fig. 4). A higher coverage of interview
fragments and more participants, particularly participants loading
significantly on Viewpoint B and C, would likely have resulted in a more
consistent image and more understanding of the participant's underlying
reasoning. Future research should consider more time for post-sorting
interviews or organizing group discussions.</p>
      <p>The viewpoints identified in this study are expected to be representative of
water authorities in the Netherlands. Only 8 out of 24 Dutch regional water
authorities were included, so additional viewpoints may be found if the study
would be repeated with participants from the remaining water authorities. We
consider this unlikely as none of the selection criteria (see Stage 3 in the
method section) were found to influence the results. Flood risk, age (years
since the last organizational reform) and location were incorporated as
characteristics that might influence participant's viewpoints. All viewpoints
represented a combination of age and location; thus, there seems to be no
relationship between these characteristics and the results. Sea level might,
as all people with Viewpoint B were working at an authority responsible for
an area below sea level. Job type did not influence the results; gender
might, as almost all people with Viewpoint B were male. This study cannot
make any claims regarding the influence of flood risk or gender, but this is
a promising direction for further research.</p>
      <p>A quantitative follow-up study can be used to determine the distribution of
viewpoints across water authorities or within one organization. Such a study
would further justify generalization of the results to the Dutch water
authorities and would allow one to check how widespread the overall positive
attitude towards citizen science found in this research actually is. This
article presents results for practitioners in the Netherlands, but we
encourage others to repeat the study, particularly in other countries facing
low citizen water awareness as described by the OECD (OECD, 2011). The
developed set of statements can serve as a basis for such a study as they are
not unique to the Dutch situation.</p>
</sec>
</sec>
<sec id="Ch1.S5" sec-type="conclusions">
  <title>Conclusion</title>
      <p>This study contributes to understanding about government practitioners'
acceptance and perception of citizen science. A Q methodological approach
was applied to identify the viewpoints of practitioners on citizen science,
in the case of water quality monitoring at Dutch regional water authorities.
Water authority employees sorted a set of 46 statements related to citizen
science. Three factors were identified in a factor analysis and translated
into corresponding viewpoint narratives.</p>
      <p>The first viewpoint, Viewpoint A, is named “Citizen participation for data
application”. People with Viewpoint A see more opportunities than challenges
when it comes to citizen science. They see applications in practical use of
the data, but also in the active engagement of people. The second viewpoint,
Viewpoint B, is named “Water authority in control”. People with this
viewpoint see a potential for data contributions by citizens in an
illustrative way, but are concerned with challenges in organizational
capacity, expectation management and motivating citizens as well. The third
viewpoint, Viewpoint C, is named “Education and local knowledge”. People
with this viewpoint focus on educational goals, such as teaching people about
their environment and getting schools involved. They consider data
applicability of secondary importance, although the data can be used
illustratively.</p>
      <p>The outcomes of this study provide strong indications that practitioners at
the Dutch water authorities welcome citizen science. These practitioners
further believe citizen science can contribute to bridging the awareness gap
as identified by the OECD (2014) and the Dutch water authorities (UvW,
2015a). All three viewpoints are positive towards citizen science in the form
of contributory projects in which citizens collect data. People with
Viewpoint A support collaborative citizen science as well, but none of the
viewpoints support co-created projects between citizens and water
authorities. Interviews identified low expectations in citizens' motivations
and capacities as underlying causes of this low support for higher levels of
citizen involvement. This may jeopardize the much-needed trust in the
relationship between citizens and practitioners. Although participants
recognized the potential of citizen science to change governance structures,
the design of citizen science projects in the Netherlands is not expected to
move beyond contributory projects in the near future.</p>
</sec>

      
      </body>
    <back><ack><title>Acknowledgements</title><p>We thank S. Abu Shoaib and an anonymous referee for their valuable comments
on the discussion paper.<?xmltex \hack{\newline}?><?xmltex \hack{\newline}?>
Edited by: S. Illingworth<?xmltex \hack{\newline}?>
Reviewed by:  S. Abu Shoaib and one anonymous referee</p></ack><ref-list>
    <title>References</title>

      <ref id="bib1.bib1"><label>1</label><mixed-citation>Bonney, R., Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T.,
Rosenberg, K. V., and Shirk, J.: Citizen Science: A Developing Tool
for Expanding Science Knowledge and Scientific Literacy,
BioScience, 59, 977–984,
<ext-link xlink:href="http://dx.doi.org/10.1525/bio.2009.59.11.9" ext-link-type="DOI">10.1525/bio.2009.59.11.9</ext-link>, 2009.</mixed-citation></ref>
      <ref id="bib1.bib2"><label>2</label><mixed-citation>
Brown, S. R.:  Political subjectivity: Applicaitons of Q
methodology in political science, New Haven, CT, Yale University Press, 1980.</mixed-citation></ref>
      <ref id="bib1.bib3"><label>3</label><mixed-citation>Buytaert, W., Zulkafli, Z., Grainger, S., Acosta, L., Alemie, T. C.,
Bastiaensen, J., De Bièvre, B., Bhusal, J., Clark, J., Dewulf, A.,
Foggin, M., Hannah, D., Hergarten, C., Isaeva, A., Karpouzoglou, T., Pandeya,
B., Paudel, D., Sharma, K., Steenhuis, T., Tilahun, S., Van Hecken, G., and
Zhumanova, M.: Citizen science in hydrology and water resources:
opportunities for knowledge generation, ecosystem service management, and
sustainable development, Front. Earth Sci., 2, 1–21,
<ext-link xlink:href="http://dx.doi.org/10.3389/feart.2014.00026" ext-link-type="DOI">10.3389/feart.2014.00026</ext-link>, 2014.</mixed-citation></ref>
      <ref id="bib1.bib4"><label>4</label><mixed-citation>Chandler, D. and Kapelner, A.: Breaking monotony with meaning:
Motivation in crowdsourcing markets, J. Econ. Behav.
Organ., 90, 123–133,
<ext-link xlink:href="http://dx.doi.org/10.1016/j.jebo.2013.03.003" ext-link-type="DOI">10.1016/j.jebo.2013.03.003</ext-link>, 2013.</mixed-citation></ref>
      <ref id="bib1.bib5"><label>5</label><mixed-citation>Clark, M. P., Nijssen, B., Lundquist, J. D., Kavetski, D., Rupp, D. E.,
Woods, R. A., Freer, J. E., Gutmann, E. D., Wood, A. W., Brekke, L. D.,
Arnold, J. R., Gochis, D. J., and Rasmussen, R. M.: A unified approach for
process-based hydrologic modeling: 1. Modeling concept, Water Resour. Res.,
51, 2498–2514, <ext-link xlink:href="http://dx.doi.org/10.1002/2015WR017200" ext-link-type="DOI">10.1002/2015WR017200</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bib6"><label>6</label><mixed-citation>Cohn, J. P.: Citizen Science?: Can Volunteers Do Real Research?,
BioScience, 58, 192–197,
<ext-link xlink:href="http://dx.doi.org/10.1641/B580303" ext-link-type="DOI">10.1641/B580303</ext-link>, 2008.</mixed-citation></ref>
      <ref id="bib1.bib7"><label>7</label><mixed-citation>Conrad, C. C. and Hilchey, K. G.: A review of citizen science and
community-based environmental monitoring: Issues and opportunities,
Environ. Monit. Assess., 176,
273–291, <ext-link xlink:href="http://dx.doi.org/10.1007/s10661-010-1582-5" ext-link-type="DOI">10.1007/s10661-010-1582-5</ext-link>, 2011.</mixed-citation></ref>
      <ref id="bib1.bib8"><label>8</label><mixed-citation>Cools, M., Moons, E., Janssens, B., and Wets, G.: Shifting towards
environment-friendly modes: Profiling travelers using Q-methodology,
Transportation, 36, 437–453,
<ext-link xlink:href="http://dx.doi.org/10.1007/s11116-009-9206-z" ext-link-type="DOI">10.1007/s11116-009-9206-z</ext-link>, 2009.</mixed-citation></ref>
      <ref id="bib1.bib9"><label>9</label><mixed-citation>Cooper, C., Dickinson, J., Phillips, T., and Bonney, R.: Citizen Science as a
Tool for Conservation in Residential Ecosystems, Ecol. Soc., 12, 11,
<ext-link xlink:href="http://dx.doi.org/10.5751/ES-02197-120211" ext-link-type="DOI">10.5751/ES-02197-120211</ext-link>, 2007.</mixed-citation></ref>
      <ref id="bib1.bib10"><label>10</label><mixed-citation>Fraternali, P., Castelletti, A., Soncini-Sessa, R., Vaca Ruiz, C., and
Rizzoli, A. E.: Putting humans in the loop: Social computing for
Water Resources Management, Environ. Modell. Softw.,
37, 68–77, <ext-link xlink:href="http://dx.doi.org/10.1016/j.envsoft.2012.03.002" ext-link-type="DOI">10.1016/j.envsoft.2012.03.002</ext-link>, 2012.</mixed-citation></ref>
      <ref id="bib1.bib11"><label>11</label><mixed-citation>
Gallagher, K. and Porock, D.: The use of interviews in Q
methodology: card content analysis, Nurs. Res.,
59, 295–300, 2010.</mixed-citation></ref>
      <ref id="bib1.bib12"><label>12</label><mixed-citation>Gharesifard, M. and Wehn, U.: To share or not to share: Drivers and
barriers for sharing data via online amateur weather networks,
J. Hydrol., 535, 181–190, <ext-link xlink:href="http://dx.doi.org/10.1016/j.jhydrol.2016.01.036" ext-link-type="DOI">10.1016/j.jhydrol.2016.01.036</ext-link>, 2016.</mixed-citation></ref>
      <ref id="bib1.bib13"><label>13</label><mixed-citation>Hobbs, S. J. and White, P. C. L.: Motivations and barriers in
relation to community participation in biodiversity recording.
J. Nat. Conserv., 20, 364–373,
<ext-link xlink:href="http://dx.doi.org/10.1016/j.jnc.2012.08.002" ext-link-type="DOI">10.1016/j.jnc.2012.08.002</ext-link>, 2012.</mixed-citation></ref>
      <ref id="bib1.bib14"><label>14</label><mixed-citation>Hollow, B., Roetman, P. E. J., Walter, M., and Daniels, C. B.:
Citizen science for policy development: The case of koala management in
South Australia. Environ. Sci. Policy, 47,
126–136, <ext-link xlink:href="http://dx.doi.org/10.1016/j.envsci.2014.10.007" ext-link-type="DOI">10.1016/j.envsci.2014.10.007</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bib15"><label>15</label><mixed-citation>Hut, R., Amisigo, B. A., Steele-Dunne, S., and van de Giesen, N.:
Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF): A data
assimilation scheme for memory intensive, high performance computing,
Adv. Water Resour., 86, 273–283, <ext-link xlink:href="http://dx.doi.org/10.1016/j.advwatres.2015.09.007" ext-link-type="DOI">10.1016/j.advwatres.2015.09.007</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bib16"><label>16</label><mixed-citation>Land-Zandstra, A. M., Devilee, J. L. A., Snik, F., Buurmeijer, F., and van
den Broek, J. M.: Citizen science on a smartphone: Participants'
motivations and learning, Public Understanding of Science (Bristol,
England), November, <ext-link xlink:href="http://dx.doi.org/10.1177/0963662515602406" ext-link-type="DOI">10.1177/0963662515602406</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bib17"><label>17</label><mixed-citation>Macknick, J. E. and Enders, S. K.: Transboundary Forestry and Water
Management in Nicaragua and Honduras: From Conflicts to Opportunities for
Cooperation, Journal of Sustainable Forestry, 31,
376–395, <ext-link xlink:href="http://dx.doi.org/10.1080/10549811.2011.588473" ext-link-type="DOI">10.1080/10549811.2011.588473</ext-link>, 2012.</mixed-citation></ref>
      <ref id="bib1.bib18"><label>18</label><mixed-citation>OECD: Water Governance in OECD Countries: A Multi-level Approach, OECD
Studies on Water series, OECD Publishing: Organisation of Economic
Cooperation and Development, <ext-link xlink:href="http://dx.doi.org/10.1787/9789264119284-en" ext-link-type="DOI">10.1787/9789264119284-en</ext-link>, 2011.</mixed-citation></ref>
      <ref id="bib1.bib19"><label>19</label><mixed-citation>OECD: Water Governance in the Netherlands, OECD Publishing,
<ext-link xlink:href="http://dx.doi.org/10.1787/9789264102637-en" ext-link-type="DOI">10.1787/9789264102637-en</ext-link>,  2014.</mixed-citation></ref>
      <ref id="bib1.bib20"><label>20</label><mixed-citation>
Ottinger, G.: Buckets of resistance: Standards and the effectiveness
of citizen science, Sci. Technol. Hum. Val., 35, 244–270, 2010.</mixed-citation></ref>
      <ref id="bib1.bib21"><label>21</label><mixed-citation>Raadgever, G. T., Mostert, E., and van de Giesen, N. C.: Identification of
stakeholder perspectives on future flood management in the Rhine basin using
Q methodology, Hydrol. Earth Syst. Sci., 12, 1097–1109,
<ext-link xlink:href="http://dx.doi.org/10.5194/hess-12-1097-2008" ext-link-type="DOI">10.5194/hess-12-1097-2008</ext-link>, 2008.</mixed-citation></ref>
      <ref id="bib1.bib22"><label>22</label><mixed-citation>Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P.,
Schawinski, K., Szalay, A. S., and Vandenberg, J.: Galaxy Zoo: Exploring the
Motivations of Citizen Science Volunteers, Astronomy Education Review, 9,
010103-1, <ext-link xlink:href="http://dx.doi.org/10.3847/AER2009036" ext-link-type="DOI">10.3847/AER2009036</ext-link>, 2010.</mixed-citation></ref>
      <ref id="bib1.bib23"><label>23</label><mixed-citation>
Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J., and Vukovic,
M.: An Assessment of Intrinsic and Extrinsic Motivation on Task Performance
in Crowdsourcing Markets, Fifth International AAAI Conference on Weblogs and
Social Media, (Gibbons 1997), 321–328, 2011.</mixed-citation></ref>
      <ref id="bib1.bib24"><label>24</label><mixed-citation>Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C.,
Lewis, D., and Jacobs, D.: Dynamic Changes in Motivation in Collaborative
Citizen-Science Projects, Proceedings of the ACM 2012 Conference on Computer
Supported Cooperative Work – CSCW'12, 217–226,
<ext-link xlink:href="http://dx.doi.org/10.1145/2145204.2145238" ext-link-type="DOI">10.1145/2145204.2145238</ext-link>, 2012.</mixed-citation></ref>
      <ref id="bib1.bib25"><label>25</label><mixed-citation>Roy, H. E., Pocock, M. J. O., Preston, C. D., Roy, D. B., Savage, J.,
Tweddle, J. C., and Robinson, L. D.: Understanding Citizen Science &amp;
Environmental Monitoring, Final Report on behalf of UK-Environmental
Observation Framework, 170, <uri>http://nora.nerc.ac.uk/20679/</uri> (last access:
31 August 2016), 2012.
</mixed-citation></ref><?xmltex \hack{\newpage}?>
      <ref id="bib1.bib26"><label>26</label><mixed-citation>Shoaib, S. A., Marshall, L., and Sharma, A.: A metric for attributing
variability in modelled streamflows, J. Hydrology, 541, 1475–1487,
<ext-link xlink:href="http://dx.doi.org/10.1016/j.jhydrol.2016.08.050" ext-link-type="DOI">10.1016/j.jhydrol.2016.08.050</ext-link>, 2016.</mixed-citation></ref>
      <ref id="bib1.bib27"><label>27</label><mixed-citation>Silvertown, J.: A new dawn for citizen science, Trends Ecol. Evol., 24,
467–471, <ext-link xlink:href="http://dx.doi.org/10.1016/j.tree.2009.03.017" ext-link-type="DOI">10.1016/j.tree.2009.03.017</ext-link>, 2009.</mixed-citation></ref>
      <ref id="bib1.bib28"><label>28</label><mixed-citation>Snik, F., Rietjens, J. H. H., Apituley, A., Volten, H., Mijling, B., Di Noia,
A., Heikamp, S., Heinsbroek, R. C., Hasekamp, O. P., Smit, J. M., Vonk, J.,
Stam, D. M., Van Harten, G., De Boer, J., and Keller, C. U.:
Mapping atmospheric aerosols with a citizen science network of
smartphone spectropolarimeters, Geophys. Res. Lett., 41, 7351–7358,
<ext-link xlink:href="http://dx.doi.org/10.1002/2014GL061462" ext-link-type="DOI">10.1002/2014GL061462</ext-link>, 2014.</mixed-citation></ref>
      <ref id="bib1.bib29"><label>29</label><mixed-citation>
Tielrooij, F.: Aders omgaan met water: waterbeleid voor de 21e eeuw [Dealing
differently with water: water policy for the 21st century], Ministry of
Public Works, Transport and Water Management, Den Haag, 2000.</mixed-citation></ref>
      <ref id="bib1.bib30"><label>30</label><mixed-citation>Tulloch, A. I. T., Possingham, H. P., Joseph, L. N., Szabo, J., and Martin,
T. G.: Realising the full potential of citizen science monitoring programs,
Biol. Conserv., 165, 128–138, <ext-link xlink:href="http://dx.doi.org/10.1016/j.biocon.2013.05.025" ext-link-type="DOI">10.1016/j.biocon.2013.05.025</ext-link>, 2013.</mixed-citation></ref>
      <ref id="bib1.bib31"><label>31</label><mixed-citation>UvW: Visie openbaar bestuur “waterbestuur dat werkt” [Vison on governance:
“water management that works”], Unie van Waterschappen [Dutch Water
Authorities], Den Haag,
<uri>https://www.uvw.nl/wp-content/uploads/2015/07/Waterbestuur-dat-werkt-2015.pdf</uri>
(last access: 12 March 2016), 2015a.</mixed-citation></ref>
      <ref id="bib1.bib32"><label>32</label><mixed-citation>UvW: Waterbeheer doen we samen: waterschappen voor de burger [We do water
management together: water authorities for citizens], Unie van Waterschappen
[Dutch Water Authorities], Den Haag,
<uri>www.uvw.nl/wp-content/uploads/2015/07/Waterschappen-voor-de-burger-2015.pdf</uri>
(last access: 12 March 2016), 2015b.</mixed-citation></ref>
      <ref id="bib1.bib33"><label>33</label><mixed-citation>
Van der Auweraert, A.: The Science Communication Escalator, in: 2nd
International Living Knowledge Conference Seville, Spain, 3–5 February 2005,
237–241, 2005.</mixed-citation></ref>
      <ref id="bib1.bib34"><label>34</label><mixed-citation>
Van Exel, J. and De Graaf, G.: Q methodology?: A sneak preview, Soc. Sci.,
2, 1–30, 2005.</mixed-citation></ref>
      <ref id="bib1.bib35"><label>35</label><mixed-citation>Varner, J.: Scientific Outreach: Toward Effective Public Engagement with
Biological Science, BioScience, 64, 333–340, <ext-link xlink:href="http://dx.doi.org/10.1093/biosci/biu021" ext-link-type="DOI">10.1093/biosci/biu021</ext-link>,
2014.</mixed-citation></ref>
      <ref id="bib1.bib36"><label>36</label><mixed-citation>
Watts, S. and Stenner, P.: Doing Q methodological research: theory, method
and interpretation, SAGE Publications, London, 2012.</mixed-citation></ref>
      <ref id="bib1.bib37"><label>37</label><mixed-citation>Wehn, U. and Evers, J.: Citizen observatories of water: Social Innovation via
eParticipation?, 2nd International Conference on ICT for Sustainability,
(Ict4s), 10.,
<uri>http://www.atlantis-press.com/php/download_paper.php?id=13419</uri> (last
access: 29 August 2016), 2014.</mixed-citation></ref>
      <ref id="bib1.bib38"><label>38</label><mixed-citation>Wehn, U., Rusca, M., Evers, J., and Lanfranchi, V.: Participation in flood
risk management and the potential of citizen observatories: A governance
analysis, Environ. Sci. Policy, 48, 225–236,
<ext-link xlink:href="http://dx.doi.org/10.1016/j.envsci.2014.12.017" ext-link-type="DOI">10.1016/j.envsci.2014.12.017</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bib39"><label>39</label><mixed-citation>Weng, Y. C.: Contrasting visions of science in ecological restoration:
Expert-lay dynamics between professional practitioners and volunteers,
Geoforum, 65, 134–145, <ext-link xlink:href="http://dx.doi.org/10.1016/j.geoforum.2015.07.023" ext-link-type="DOI">10.1016/j.geoforum.2015.07.023</ext-link>, 2015.</mixed-citation></ref>

  </ref-list><app-group content-type="float"><app><title/>

    </app></app-group></back>
    <!--<article-title-html>Practitioners' viewpoints on citizen science in water management: a case study in Dutch regional water resource management</article-title-html>
<abstract-html><p class="p">In recent years, governmental institutes have started to use citizen science
as a form of public participation. The Dutch water authorities are among
them. They face pressure on the water governance system and a water awareness
gap among the general public, and consider citizen science a possible
solution. The reasons for practitioners to engage in citizen science, and in
particular those of government practitioners, have seldom been studied. This
article aims to pinpoint the various viewpoints of practitioners at Dutch
regional water authorities on citizen science. A Q-methodological approach
was used because it allows for exploration of viewpoints and statistical
analysis using a small sample size. Practitioners (33) at eight different
water authorities ranked 46 statements from agree to disagree. Three
viewpoints were identified with a total explained variance of 67 %.
Viewpoint A considers citizen science a potential solution that can serve
several purposes, thereby encouraging citizen participation in data
collection and analysis. Viewpoint B considers citizen science a method for
additional, illustrative data. Viewpoint C views citizen science primarily as
a means of education. These viewpoints show water practitioners in the
Netherlands are willing to embrace citizen science at water authorities,
although there is no support for higher levels of citizen engagement.</p></abstract-html>
<ref-html id="bib1.bib1"><label>1</label><mixed-citation>
Bonney, R., Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T.,
Rosenberg, K. V., and Shirk, J.: Citizen Science: A Developing Tool
for Expanding Science Knowledge and Scientific Literacy,
BioScience, 59, 977–984,
<a href="http://dx.doi.org/10.1525/bio.2009.59.11.9" target="_blank">doi:10.1525/bio.2009.59.11.9</a>, 2009.
</mixed-citation></ref-html>
<ref-html id="bib1.bib2"><label>2</label><mixed-citation>
Brown, S. R.:  Political subjectivity: Applicaitons of Q
methodology in political science, New Haven, CT, Yale University Press, 1980.
</mixed-citation></ref-html>
<ref-html id="bib1.bib3"><label>3</label><mixed-citation>
Buytaert, W., Zulkafli, Z., Grainger, S., Acosta, L., Alemie, T. C.,
Bastiaensen, J., De Bièvre, B., Bhusal, J., Clark, J., Dewulf, A.,
Foggin, M., Hannah, D., Hergarten, C., Isaeva, A., Karpouzoglou, T., Pandeya,
B., Paudel, D., Sharma, K., Steenhuis, T., Tilahun, S., Van Hecken, G., and
Zhumanova, M.: Citizen science in hydrology and water resources:
opportunities for knowledge generation, ecosystem service management, and
sustainable development, Front. Earth Sci., 2, 1–21,
<a href="http://dx.doi.org/10.3389/feart.2014.00026" target="_blank">doi:10.3389/feart.2014.00026</a>, 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib4"><label>4</label><mixed-citation>
Chandler, D. and Kapelner, A.: Breaking monotony with meaning:
Motivation in crowdsourcing markets, J. Econ. Behav.
Organ., 90, 123–133,
<a href="http://dx.doi.org/10.1016/j.jebo.2013.03.003" target="_blank">doi:10.1016/j.jebo.2013.03.003</a>, 2013.
</mixed-citation></ref-html>
<ref-html id="bib1.bib5"><label>5</label><mixed-citation>
Clark, M. P., Nijssen, B., Lundquist, J. D., Kavetski, D., Rupp, D. E.,
Woods, R. A., Freer, J. E., Gutmann, E. D., Wood, A. W., Brekke, L. D.,
Arnold, J. R., Gochis, D. J., and Rasmussen, R. M.: A unified approach for
process-based hydrologic modeling: 1. Modeling concept, Water Resour. Res.,
51, 2498–2514, <a href="http://dx.doi.org/10.1002/2015WR017200" target="_blank">doi:10.1002/2015WR017200</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib6"><label>6</label><mixed-citation>
Cohn, J. P.: Citizen Science?: Can Volunteers Do Real Research?,
BioScience, 58, 192–197,
<a href="http://dx.doi.org/10.1641/B580303" target="_blank">doi:10.1641/B580303</a>, 2008.
</mixed-citation></ref-html>
<ref-html id="bib1.bib7"><label>7</label><mixed-citation>
Conrad, C. C. and Hilchey, K. G.: A review of citizen science and
community-based environmental monitoring: Issues and opportunities,
Environ. Monit. Assess., 176,
273–291, <a href="http://dx.doi.org/10.1007/s10661-010-1582-5" target="_blank">doi:10.1007/s10661-010-1582-5</a>, 2011.
</mixed-citation></ref-html>
<ref-html id="bib1.bib8"><label>8</label><mixed-citation>
Cools, M., Moons, E., Janssens, B., and Wets, G.: Shifting towards
environment-friendly modes: Profiling travelers using Q-methodology,
Transportation, 36, 437–453,
<a href="http://dx.doi.org/10.1007/s11116-009-9206-z" target="_blank">doi:10.1007/s11116-009-9206-z</a>, 2009.
</mixed-citation></ref-html>
<ref-html id="bib1.bib9"><label>9</label><mixed-citation>
Cooper, C., Dickinson, J., Phillips, T., and Bonney, R.: Citizen Science as a
Tool for Conservation in Residential Ecosystems, Ecol. Soc., 12, 11,
<a href="http://dx.doi.org/10.5751/ES-02197-120211" target="_blank">doi:10.5751/ES-02197-120211</a>, 2007.
</mixed-citation></ref-html>
<ref-html id="bib1.bib10"><label>10</label><mixed-citation>
Fraternali, P., Castelletti, A., Soncini-Sessa, R., Vaca Ruiz, C., and
Rizzoli, A. E.: Putting humans in the loop: Social computing for
Water Resources Management, Environ. Modell. Softw.,
37, 68–77, <a href="http://dx.doi.org/10.1016/j.envsoft.2012.03.002" target="_blank">doi:10.1016/j.envsoft.2012.03.002</a>, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib11"><label>11</label><mixed-citation>
Gallagher, K. and Porock, D.: The use of interviews in Q
methodology: card content analysis, Nurs. Res.,
59, 295–300, 2010.
</mixed-citation></ref-html>
<ref-html id="bib1.bib12"><label>12</label><mixed-citation>
Gharesifard, M. and Wehn, U.: To share or not to share: Drivers and
barriers for sharing data via online amateur weather networks,
J. Hydrol., 535, 181–190, <a href="http://dx.doi.org/10.1016/j.jhydrol.2016.01.036" target="_blank">doi:10.1016/j.jhydrol.2016.01.036</a>, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib13"><label>13</label><mixed-citation>
Hobbs, S. J. and White, P. C. L.: Motivations and barriers in
relation to community participation in biodiversity recording.
J. Nat. Conserv., 20, 364–373,
<a href="http://dx.doi.org/10.1016/j.jnc.2012.08.002" target="_blank">doi:10.1016/j.jnc.2012.08.002</a>, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib14"><label>14</label><mixed-citation>
Hollow, B., Roetman, P. E. J., Walter, M., and Daniels, C. B.:
Citizen science for policy development: The case of koala management in
South Australia. Environ. Sci. Policy, 47,
126–136, <a href="http://dx.doi.org/10.1016/j.envsci.2014.10.007" target="_blank">doi:10.1016/j.envsci.2014.10.007</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib15"><label>15</label><mixed-citation>
Hut, R., Amisigo, B. A., Steele-Dunne, S., and van de Giesen, N.:
Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF): A data
assimilation scheme for memory intensive, high performance computing,
Adv. Water Resour., 86, 273–283, <a href="http://dx.doi.org/10.1016/j.advwatres.2015.09.007" target="_blank">doi:10.1016/j.advwatres.2015.09.007</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib16"><label>16</label><mixed-citation>
Land-Zandstra, A. M., Devilee, J. L. A., Snik, F., Buurmeijer, F., and van
den Broek, J. M.: Citizen science on a smartphone: Participants'
motivations and learning, Public Understanding of Science (Bristol,
England), November, <a href="http://dx.doi.org/10.1177/0963662515602406" target="_blank">doi:10.1177/0963662515602406</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib17"><label>17</label><mixed-citation>
Macknick, J. E. and Enders, S. K.: Transboundary Forestry and Water
Management in Nicaragua and Honduras: From Conflicts to Opportunities for
Cooperation, Journal of Sustainable Forestry, 31,
376–395, <a href="http://dx.doi.org/10.1080/10549811.2011.588473" target="_blank">doi:10.1080/10549811.2011.588473</a>, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib18"><label>18</label><mixed-citation>
OECD: Water Governance in OECD Countries: A Multi-level Approach, OECD
Studies on Water series, OECD Publishing: Organisation of Economic
Cooperation and Development, <a href="http://dx.doi.org/10.1787/9789264119284-en" target="_blank">doi:10.1787/9789264119284-en</a>, 2011.
</mixed-citation></ref-html>
<ref-html id="bib1.bib19"><label>19</label><mixed-citation>
OECD: Water Governance in the Netherlands, OECD Publishing,
<a href="http://dx.doi.org/10.1787/9789264102637-en" target="_blank">doi:10.1787/9789264102637-en</a>,  2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib20"><label>20</label><mixed-citation>
Ottinger, G.: Buckets of resistance: Standards and the effectiveness
of citizen science, Sci. Technol. Hum. Val., 35, 244–270, 2010.
</mixed-citation></ref-html>
<ref-html id="bib1.bib21"><label>21</label><mixed-citation>
Raadgever, G. T., Mostert, E., and van de Giesen, N. C.: Identification of
stakeholder perspectives on future flood management in the Rhine basin using
Q methodology, Hydrol. Earth Syst. Sci., 12, 1097–1109,
<a href="http://dx.doi.org/10.5194/hess-12-1097-2008" target="_blank">doi:10.5194/hess-12-1097-2008</a>, 2008.
</mixed-citation></ref-html>
<ref-html id="bib1.bib22"><label>22</label><mixed-citation>
Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P.,
Schawinski, K., Szalay, A. S., and Vandenberg, J.: Galaxy Zoo: Exploring the
Motivations of Citizen Science Volunteers, Astronomy Education Review, 9,
010103-1, <a href="http://dx.doi.org/10.3847/AER2009036" target="_blank">doi:10.3847/AER2009036</a>, 2010.
</mixed-citation></ref-html>
<ref-html id="bib1.bib23"><label>23</label><mixed-citation>
Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J., and Vukovic,
M.: An Assessment of Intrinsic and Extrinsic Motivation on Task Performance
in Crowdsourcing Markets, Fifth International AAAI Conference on Weblogs and
Social Media, (Gibbons 1997), 321–328, 2011.
</mixed-citation></ref-html>
<ref-html id="bib1.bib24"><label>24</label><mixed-citation>
Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C.,
Lewis, D., and Jacobs, D.: Dynamic Changes in Motivation in Collaborative
Citizen-Science Projects, Proceedings of the ACM 2012 Conference on Computer
Supported Cooperative Work – CSCW'12, 217–226,
<a href="http://dx.doi.org/10.1145/2145204.2145238" target="_blank">doi:10.1145/2145204.2145238</a>, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib25"><label>25</label><mixed-citation>
Roy, H. E., Pocock, M. J. O., Preston, C. D., Roy, D. B., Savage, J.,
Tweddle, J. C., and Robinson, L. D.: Understanding Citizen Science &amp;
Environmental Monitoring, Final Report on behalf of UK-Environmental
Observation Framework, 170, <a href="http://nora.nerc.ac.uk/20679/" target="_blank">http://nora.nerc.ac.uk/20679/</a> (last access:
31 August 2016), 2012.

</mixed-citation></ref-html>
<ref-html id="bib1.bib26"><label>26</label><mixed-citation>
Shoaib, S. A., Marshall, L., and Sharma, A.: A metric for attributing
variability in modelled streamflows, J. Hydrology, 541, 1475–1487,
<a href="http://dx.doi.org/10.1016/j.jhydrol.2016.08.050" target="_blank">doi:10.1016/j.jhydrol.2016.08.050</a>, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib27"><label>27</label><mixed-citation>
Silvertown, J.: A new dawn for citizen science, Trends Ecol. Evol., 24,
467–471, <a href="http://dx.doi.org/10.1016/j.tree.2009.03.017" target="_blank">doi:10.1016/j.tree.2009.03.017</a>, 2009.
</mixed-citation></ref-html>
<ref-html id="bib1.bib28"><label>28</label><mixed-citation>
Snik, F., Rietjens, J. H. H., Apituley, A., Volten, H., Mijling, B., Di Noia,
A., Heikamp, S., Heinsbroek, R. C., Hasekamp, O. P., Smit, J. M., Vonk, J.,
Stam, D. M., Van Harten, G., De Boer, J., and Keller, C. U.:
Mapping atmospheric aerosols with a citizen science network of
smartphone spectropolarimeters, Geophys. Res. Lett., 41, 7351–7358,
<a href="http://dx.doi.org/10.1002/2014GL061462" target="_blank">doi:10.1002/2014GL061462</a>, 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib29"><label>29</label><mixed-citation>
Tielrooij, F.: Aders omgaan met water: waterbeleid voor de 21e eeuw [Dealing
differently with water: water policy for the 21st century], Ministry of
Public Works, Transport and Water Management, Den Haag, 2000.
</mixed-citation></ref-html>
<ref-html id="bib1.bib30"><label>30</label><mixed-citation>
Tulloch, A. I. T., Possingham, H. P., Joseph, L. N., Szabo, J., and Martin,
T. G.: Realising the full potential of citizen science monitoring programs,
Biol. Conserv., 165, 128–138, <a href="http://dx.doi.org/10.1016/j.biocon.2013.05.025" target="_blank">doi:10.1016/j.biocon.2013.05.025</a>, 2013.
</mixed-citation></ref-html>
<ref-html id="bib1.bib31"><label>31</label><mixed-citation>
UvW: Visie openbaar bestuur “waterbestuur dat werkt” [Vison on governance:
“water management that works”], Unie van Waterschappen [Dutch Water
Authorities], Den Haag,
<a href="https://www.uvw.nl/wp-content/uploads/2015/07/Waterbestuur-dat-werkt-2015.pdf" target="_blank">https://www.uvw.nl/wp-content/uploads/2015/07/Waterbestuur-dat-werkt-2015.pdf</a>
(last access: 12 March 2016), 2015a.
</mixed-citation></ref-html>
<ref-html id="bib1.bib32"><label>32</label><mixed-citation>
UvW: Waterbeheer doen we samen: waterschappen voor de burger [We do water
management together: water authorities for citizens], Unie van Waterschappen
[Dutch Water Authorities], Den Haag,
<a href="www.uvw.nl/wp-content/uploads/2015/07/Waterschappen-voor-de-burger-2015.pdf" target="_blank">www.uvw.nl/wp-content/uploads/2015/07/Waterschappen-voor-de-burger-2015.pdf</a>
(last access: 12 March 2016), 2015b.
</mixed-citation></ref-html>
<ref-html id="bib1.bib33"><label>33</label><mixed-citation>
Van der Auweraert, A.: The Science Communication Escalator, in: 2nd
International Living Knowledge Conference Seville, Spain, 3–5 February 2005,
237–241, 2005.
</mixed-citation></ref-html>
<ref-html id="bib1.bib34"><label>34</label><mixed-citation>
Van Exel, J. and De Graaf, G.: Q methodology?: A sneak preview, Soc. Sci.,
2, 1–30, 2005.
</mixed-citation></ref-html>
<ref-html id="bib1.bib35"><label>35</label><mixed-citation>
Varner, J.: Scientific Outreach: Toward Effective Public Engagement with
Biological Science, BioScience, 64, 333–340, <a href="http://dx.doi.org/10.1093/biosci/biu021" target="_blank">doi:10.1093/biosci/biu021</a>,
2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib36"><label>36</label><mixed-citation>
Watts, S. and Stenner, P.: Doing Q methodological research: theory, method
and interpretation, SAGE Publications, London, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib37"><label>37</label><mixed-citation>
Wehn, U. and Evers, J.: Citizen observatories of water: Social Innovation via
eParticipation?, 2nd International Conference on ICT for Sustainability,
(Ict4s), 10.,
<a href="http://www.atlantis-press.com/php/download_paper.php?id=13419" target="_blank">http://www.atlantis-press.com/php/download_paper.php?id=13419</a> (last
access: 29 August 2016), 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib38"><label>38</label><mixed-citation>
Wehn, U., Rusca, M., Evers, J., and Lanfranchi, V.: Participation in flood
risk management and the potential of citizen observatories: A governance
analysis, Environ. Sci. Policy, 48, 225–236,
<a href="http://dx.doi.org/10.1016/j.envsci.2014.12.017" target="_blank">doi:10.1016/j.envsci.2014.12.017</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib39"><label>39</label><mixed-citation>
Weng, Y. C.: Contrasting visions of science in ecological restoration:
Expert-lay dynamics between professional practitioners and volunteers,
Geoforum, 65, 134–145, <a href="http://dx.doi.org/10.1016/j.geoforum.2015.07.023" target="_blank">doi:10.1016/j.geoforum.2015.07.023</a>, 2015.
</mixed-citation></ref-html>--></article>
