Previous research has suggested that the use of more authentic learning activities can produce more robust and durable knowledge gains. This is consistent with calls within civil engineering education, specifically hydrology, that suggest that curricula should more often include professional perspective and data analysis skills to better develop the “T-shaped” knowledge profile of a professional hydrologist (i.e., professional breadth combined with technical depth). It was expected that the inclusion of a data-driven simulation lab exercise that was contextualized within a real-world situation and more consistent with the job duties of a professional in the field, would provide enhanced learning and appreciation of job duties beyond more conventional paper-and-pencil exercises in a lower-division undergraduate course. Results indicate that while students learned in both conditions, learning was enhanced for the data-driven simulation group in nearly every content area. This pattern of results suggests that the use of data-driven modeling and visualization activities can have a significant positive impact on instruction. This increase in learning likely facilitates the development of student perspective and conceptual mastery, enabling students to make better choices about their studies, while also better preparing them for work as a professional in the field.
While there is a rising interest in and demand for civil engineering and hydrology education, some have suggested a widening gap between how students are instructed in hydrology, and the subsequent professional skill set required for a career as a hydrological engineer (Wagener et al., 2007). Recent research has shown a potential for great variability within the hydrological curriculum (Wagener et al., 2012). This variability includes differences in not only what conceptual material should be taught (Gleeson et al., 2012) but also how this material should be delivered pedagogically (Wagener, 2007). It has been suggested that an emerging requirement for new hydrological engineers is the ability to not only develop a well-defined knowledge base of basic hydrological concepts but also synthesize this conceptual learning with more authentic “real-world” knowledge gained from the interpretation and application of this knowledge (Merwade and Ruddell, 2012). Unfortunately, field and modeling activities are often lacking in the hydrological curriculum, at least at the undergraduate and lower-division level (ASCE, 1990; MacDonald, 1993; Nash et al., 1990; Ruddell and Wagener, 2013; Wagener et al., 2007, 2012). This is especially concerning as, unlike laboratory sciences such as physics and chemistry, hydrology is fundamentally a place-based science. It can therefore be argued that hydrologists must engage in field and modeling activities in order to fully develop the critical ability to link hydrological concepts to applications in a specific place and/or instance (Eagleson et al., 1991).
This call to integrate experiential learning with traditional classroom instruction is not new, and has been advocated in other fields of engineering (Duderstadt, 2007; Lattuca et al., 2006; Shulman, 2005), and has also been suggested more generally within the educational literature (e.g., Bransford et al., 1999; Brown et al., 1989). These suggestions are rooted in the simple tenet that when learners engage more deeply in the formation and development of relevant knowledge, the depth and quality of their understanding subsequently increases. This constructive process is integral to numerous pedagogical philosophies such as problem-based learning (PBL), guided discovery learning, and cognitive apprenticeship, to name a few (Alfieri et al., 2011; Brown et al., 1989; Collins, 1991; De Jong and Van Joolingen, 1998; Duch et al., 2001; Savery and Duffy, 1995; Wood, 2003).
While the various educational pedagogies mentioned above are different on several levels, they share at least two important unifying characteristics. Fundamentally, (1) they require the learner to be actively engaged in the learning activity in order to realize any learning benefit, and (2) they are usually situated within an authentic or “real” problem that the student must work to solve or address. Importantly, these characteristics imply that the problem is difficult enough that students must work towards a solution (i.e., they do not know the solution initially), and that each student has explicit engagement with the pursuit of this solution, as such activities are often implemented in group settings (Smith et al., 2005). It has been argued that such authentic engagement fosters a more deep conceptual understanding of the material by “anchoring” the more abstract learning material or concepts to the more accessible authentic learning scenario (CTGV, 1992; Hake, 1998). Thus, the contextualization of the material within an actual scenario increases not only retrieval cues that the learner can use to more efficiently access factual knowledge but also likely increases the durability of the knowledge base, thereby creating a more flexible state of information that could be applied appropriately in multiple instances (Hansen, 2008; Smith and Van Doren, 2004).
Active engagement in the learning process has also been suggested as a means to increase interest in the topic to be learned (Paris and Turner, 1994; Schiefele, 1991), which might also address issues of motivation within students. Traditional lecture-based instruction that forces students to work towards normative educational goals in isolation is often cited as a major complaint of engineering students, and has measurable negative effects on motivation levels (Felder et al., 1998). More authentic, problem-based activity has been shown to produce an increase in student attitudes towards the content area in general (Watters and Ginns, 2000), offering an opportunity to offset such motivation issues. Importantly, this could not only increase motivation within the lesson itself but also potentially affect the likelihood to continue with studies in a given domain. In other words, this motivation derived within a specific context could have a direct effect on overall interest in the major or field, as learners are better able to see how their own interests better align and apply to tangible problems.
However, efforts to adopt such authentic learning exercises within engineering education are often hampered by unclear learning objectives and assessment, logistical constraints, and the use of activities that do not necessarily optimize the learning experience (see Prince, 2004). For example, it is unclear about what degree of authenticity is required, and how one can assess learning from “field activities” relative to traditional instruction. For example, while PBL has been implemented successfully with electrical engineering students (Yadav et al., 2011), students who engaged with the PBL activity were compared to students who only had a lecture component, without the opportunity to engage in an equivalent control activity. As such, these studies cannot conclusively say that gains normally attributed to the instructional manipulation are due to the activity alone, and could reflect the influence of other factors (e.g., differences in time spent engaging with the material). Further, what is an appropriate “field activity” in an engineering discipline, and how should these efforts be categorized and defined? As such, while this call for authentic activity is often advocated and supported theoretically, unfortunately it is not often consistently practiced, and thus leads to fragmented research on the issue (Prince, 2004).
There are also more specific pedagogical concerns regarding authentic learning within the area of hydrological engineering education (Gleeson et al., 2012). For example, there is little to no direct evidence that such activities are indeed effective at augmenting a hydrologist's training, or even implemented with any kind of regularity for that matter (Ruddell and Wagener, 2013). What little evidence that does exist supporting the incorporation of student-centered activities into hydrology instruction is often anecdotal (e.g., Thompson et al., 2012), without any kind of quantitative or measurable change in performance outcomes. Pragmatic and logistical issues (e.g., faculty time and expertise, student computer skills), and the use of curriculum materials that become rapidly outdated, also stand as barriers to the adoption of a more discovery-based or student-centered approach within hydrology (Merwade and Ruddell, 2012; Ruddell and Wagener, 2013). Finally, hydrological instruction is also traditionally implemented using a teacher-centered approach (e.g., lectures) that lacks the opportunity for applied experience (Wagener et al., 2007). Thus, it appears critical to find new ways to achieve instructional goals that might incorporate this real world experience, and are capable of side-stepping these methodological and logistical issues. Fortunately, the emergence of rich and dynamic computer simulation techniques, which allow students to interact with real data in ways that are consistent and appropriate with the profession, might offer an alternative to such traditional instruction, and thus provide an exciting opportunity for students to achieve this more authentic application of knowledge.
Standardized data and modeling-driven geoscience cyber-education (DMDGC) modules, developed and published by a dedicated community of educators, do potentially provide access to such dynamic and realistic learning experiences, while also avoiding some of the logistical barriers mentioned above (Habib et al., 2012; Merwade and Ruddell, 2012). These modules utilize contextually specific, rich, and dynamic computer simulations that allow students to interact with current field data in a fashion equivalent to professional hydrologists. As students do not have to physically travel to a work site to collect data, nor do they require specialized tools to work with the data, these simulation activities can be easily integrated into normal laboratory sections via coursework. Most importantly, as the data are real, and also contextualized within a specific example, it presents an opportunity to apply hydrological concepts within a formally structured and valid situation, again consistent with professionals in the field.
An open question, however, is whether such activities do in fact realize the potential educational benefits that one might anticipate from authentic activities. Similarly, it is not known for which content areas and/or aspects of the curriculum such benefits are localized or strongest, if any. For example, do such activities help students better appreciate what it means to be a professional hydrological engineer? Or is this benefit localized to better understanding hydrological domain content alone? In other words, it must be evaluated whether DMDGC modules do in fact serve as an adequate opportunity to gain such authentic activity, while also permitting the learning and achievement of traditional class goals for knowledge attainment? It has therefore become important to pinpoint the learning benefits created by DMDGC activities, so that these activities can be optimized for content, structure, and integration with the traditional lecture format.
This study directly examines the efficacy of such data-driven simulations for hydrology education at the earliest point in a potential future hydrologist's university training: in a mandatory lower-division undergraduate earth science context that is part of general curriculum studies. At this level the student enters the classroom with very little (if any) prior knowledge about hydrologic theory, hydrology models/methods, or the broad applications and societal issues involved with hydrological engineering. Evaluating the effects of such an intervention at this very early point provides an opportunity to examine the full effect of DMDGC implementation, avoiding issues of self-selection bias and prior contextual knowledge about the hydrology profession that might exist in upper division or graduate students in the field. In other words, in this student population we can observe the effect of DMDGC activities on a breadth of knowledge related to the field and its application, beyond just core theoretical concepts and applied computer modeling skills.
Students in the DMDGC condition were given a data-driven hydrology activity that focused on urbanization and flooding, while a control group was given a paper-and-pencil-based laboratory activity of equivalent general learning outcomes and effort, but lacked the specific applied context and data-driven components of the DMDGC. The inclusion of this paper-based activity is a critical methodological feature, as it permits a more appropriate evaluation of the simulation activity against an activity that is likely equally effortful and time consuming from the student perspective. Thus, any subsequent learning gains are less likely to be attributed to other confounding factors, and must instead be more localized to the nature of the manipulation itself. It was hypothesized that students who were presented with the DMDGC learning activities would demonstrate a better understanding of theoretical and applied hydrology concepts related to flooding, as their interaction with the material would be contextualized and likewise permit a dynamic exploration of the data not otherwise possible without such simulation. Further, it was also hoped that students in the simulation condition would develop a better appreciation for the roles of hydrological engineers and hydrology organizations in managing and preventing flooding problems, as they themselves are engaging in a contextualized problem within a realistic community scenario that required the intervention of hydrologists.
A total of 107 students (
Both the DMDGC modules (Ruddell and Schiesser, 2012a, b) and the comparable paper laboratory (Lab 9 in Schiesser, 2008) were designed to be implemented in parallel with traditional lectures. In this unit, all students were given identical lectures (based on material covered in Schiesser, 2008) that covered the fundamentals of flood frequency, urbanization and land use change, flood risk, climate change effects on rainfall, and the roles and responsibilities of agencies that provide flood prediction and management services in the USA. In other words, the lecture component of the current design was identical for both laboratory groups, and the only instructional difference was whether the students received a DMDGC or paper laboratory module.
Illustration of the visualization produced by the DMDGC module. The rational method and a triangular synthetic unit hydrograph are applied to model a rainstorm's streamflow hydrograph based on a rainfall input and watershed parameters, and this is visually compared with a calculated flow channel capacity to determine whether a flood will occur during a specified design storm event. The model can be calibrated such that the timing and magnitude of the observed flood peak (black diamond) matches the modeled streamflow hydrograph (blue triangle). Adapted with permission from Ruddell and Schiesser (2012).
Example of a completed assessment matrix mapping learning outcome rubrics to instrument question responses. This example student gave relevant responses to all questions (i.e., no “0” ratings), and the matrix generally indicates an “Apprentice” level of conceptual mastery.
The experimental DMDGC module is written for Microsoft Excel™, a widely utilized and highly accessible spreadsheet application. It is a simple stormwater hydrograph modeling module that applies the widely utilized rational method and a synthetic unit hydrograph to estimate hydrographs and flooding for urban watersheds. The model is based on assumptions optimized for a floodway in Maricopa County, Arizona, an urbanized desert area in the southwestern United States. The module has the ability to accept both observed rainfall and streamflow data so that a student may calibrate the parameters of the flood model to match any observed event. Importantly, the module is also broadly applicable to urbanizing watersheds anywhere in the world and can be adapted to other locations by simply adjusting a few model parameters and obtaining observed streamflow data for a flood event. As such, this DMDGC activity could be applied to nearly any urban area, an option that could be used to tailor context and content respective to each student population and their corresponding physical location.
The DMDGC module produces a visualization of modeled and observed hydrograph results (Fig. 1). As is visible in Fig. 1, the module emphasizes the determination of whether or not a given channel will flood during a 100-year design storm event as land use is progressively urbanized, and as the design storm changes due to climate change. These multiple interacting characteristics served as the foundation for the rubric described in the next section (and in Fig. 2). The DMDGC module takes roughly 2 hours of preparation for a novice instructor and roughly 3 hours of student effort to complete.
A paper lab activity (Lab 9 in Schiesser, 2008) requiring a similar effort was utilized as a control for comparison with the DMDGC module. The paper module requires students to perform hand calculations and determine whether a channel will flood before and after urbanization occurs in a watershed. Like the DMDGC module, a student considers the effect of issues such as rainfall infiltration, watershed area, rainfall intensity, and channel capacity in determining a flood. Unlike the DMDGC module, the paper activity explicitly addresses issues of flood frequency using recurrence interval calculations using a brief table of historical peak flow events instead of a student's investigation of observed streamflow data. Also, no visualization or interaction is possible with the paper method. The paper lab's streamflow data are “stock” data that are hypothetical and not drawn from real-world or place-based sources. The paper module does not include customized data for the student's local watershed, nor an observed rainfall event in the local watershed, and is not able to provide visual feedback via the flow hydrograph when the student adjusts watershed parameters or the rainfall intensity. Finally, this paper-based activity is not contextualized within the local environment (e.g., Maricopa county). The estimated time to complete this paper activity is also approximately 3 hours. Thus, this exercise requires students to complete calculations of similar complexity and type as the DMDGC module, albeit in a paper-and-pencil form and minus the place-based contextualization and interactive visualization components. As such, effort and time with the material (across both lecture and laboratory components) are comparable across these instructional conditions, and not likely explanations for any subsequent effects.
The pre-/post-assessment instrument (Table 1) features eight questions spanning a range of topics. Of these 8 questions, 2 contained sub-questions, thus resulting in a total of 11 questions overall. To provide a more coherent evaluation of performance in the learning of hydrological concepts and the role of hydrologists, a rubric was developed resulting in nine overall learning outcomes representing important hydrology concepts related to flooding. The nine outcome areas presented in Table 2 represent areas of conceptual mastery regarding climate, land cover, flood management, and hydrology. The first six areas specifically emphasize mastery of the physical concepts determining flooding (e.g., rainfall intensity and duration, hydrographs, infiltration, and stormwater management practices), and thus are indicative of a good conceptual understanding of the material itself. However, the last three outcomes were designed to assess the understanding of the roles and responsibilities of flood-related professional agencies (e.g., agency roles and responsibilities, value of geoscience knowledge), or in other words, the potential job duties of a professional working in the field. To make this distinction more transparent, examples of conceptual mastery relative to the learning outcomes are also presented in Table 2. Together, these nine outcomes cover the basic physical details and a “T-shaped profile” of professional and scientific competence (Cap-Net, 2008; McIntosh and Taylor, 2013; Pathirana et al., 2012; Pinter et al., 2013; Uhlenbrook and de Jong, 2012) needed for a basic appreciation of the profession and the social impacts of flood hydrology.
Each of the 11 questions was evaluated relative to the appropriate
learning outcomes on a four-point scale (0–3) as to the level of conceptual
mastery indicated by the response, where 0 indicates no relevant response, 1
indicates a “Novice” level, 2 indicates an “Apprentice” level, and 3
indicates an “Expert” level. A rating of 3 approximates the level of
conceptual mastery expected by a practicing hydrological professional. Some
sub-questions did not assess some learning outcomes; these irrelevant
combinations are indicated in the Fig. 2 matrix in grey. Two hydrology
educators independently coded the level of conceptual mastery indicated by
student responses on the pre-test assessment instrument, blind to condition,
and indicated a high degree of inter-rater reliability across all nine
learning outcomes (all ICCs > 0.91,
Pre-/post-assessment instrument with model “Expert” answers.
Continued.
Examples of conceptual mastery for each of nine outcomes.
To examine the effect of the DMDGC modules on the change in student knowledge
in each of the nine outcomes, a simple two-way ANCOVA was conducted between
laboratory groups on the post-test scores for each outcome. Pre-test scores
for each measure were used as a covariate in every respective analysis to
control for any differences in initial knowledge levels, and all results were
evaluated for significance at the level of
Descriptive and inferential statistics for all analyses.
PP is the paper-and-pencil group, DMDGC is the data and modeling-driven geoscience cyber-education
group;
Conceptual mastery scores for each learning objective by group. Error bars represent the standard error of the mean. (1) Recognizes that urbanization can increase impervious land cover and increase runoff and flooding, (2) explains that higher rainfall duration and intensity, combined with high soil moisture or impervious land cover, causes flooding, (3) uses correctly in context the vocabulary of land cover change, rainfall and runoff processes, and flood discharge and stage, (4) explains the effect of decadal timescale LULC change on flooding, (5) recognizes that maximum discharge rates determine flooding (hydrographs), (6) explains the tools used in flood management, (7) identifies flood management agency roles and responsibilities, (8) recognizes the value of geoscience knowledge for flood management, and (9) explains the utility of mathematical flood models.
As is visible in Table 3, the use of a DMDGC module significantly improved performance in all six areas (nos. 1–6) except for the effect of decadal LULC change on flooding (no. 4). Participants who were given the opportunity to learn with the DMDGC modules were better able to not only understand the effects of urbanization and other physical causes of flooding but also demonstrated better knowledge of maximum discharge rates and impacts of flood management. The lack of result for outcome no. 4 (the effect of decadal LULC change on flooding) was not entirely unexpected, as although this content topic was originally intended to be emphasized in the lecture and lab settings, unfortunately it was not able to be covered in depth due to time constraints. Thus, it is not surprising that this outcome showed little divergence between groups as students were not explicitly instructed in this topic. As such, this likely reflects a shortcoming in the overall content covered, rather than demonstrating a lack of theoretical effect.
Consistent with the content results above, users of DMDGC modules also appear to have gained a better appreciation for the professional role of hydrologists and the field. Across all three sub-areas (nos. 6–9), there was a significantly higher demonstration of expertise for the simulation group, above those simply using the paper-and-pencil activities. This suggests that not only does engaging in such authentic activity produce a measurable benefit in learning content but this benefit also results in a better understanding of the professional duties within the field. This result is especially encouraging as it also could potentially indicate that such activities allow students to become better prepared for eventual careers in hydrological engineering, and thus provide a bridge between the content area and the application of knowledge.
In summary, when one considers the overall pattern of results it appears that the benefit for such dynamic simulation and visualization was not only limited to content knowledge areas such as rainfall intensity and flooding but was also realized in regards to better understanding the professional and social impacts of hydrology. This suggests that not only did learners better understand the material itself but they also better understood the role of hydrologists in a more general sense. Further, the medium-to-large effect sizes (Miles and Shevlin, 2001) realized by this manipulation further suggest that the inclusion of the DMDGC module produced a practical and worthwhile change in performance, above and beyond reaching simple statistical reliability.
While prior research in education has suggested that the use of applied examples could likely benefit learning, this suggestion was explicitly tested here in the context of hydrological education, using DMDGC modules. It was anticipated that the use of such dynamic and flexible simulation tools, which enable learners to contextualize and visualize the impact of minute changes in data over time, would lead to a marked increase in learning performance. The results of this classroom study support exactly that. Learners who were permitted to interact with such simulations not only were better able to understand the content itself in the form of general knowledge but these same learners were also better able to appreciate the role of professionals within the field. This increase was significantly larger than that experienced by the control group, which engaged with materials that required similar skills but lacked the contextual and simulation components of the DMDGC module. It is our contention that the increase in both areas (breadth and depth) was a direct result of the experience with the DMDGC module. For example, in terms of conceptual learning, the DMDGC modules allowed learners to better understand the interaction of conceptual units and how to use tools like the hydrograph to anticipate flooding conditions. Similarly, this direct experience also allowed learners to better appreciate the job duties of practicing hydrologists, providing a tacit understanding of the role of agencies and geoscience education in society, which in turn led to better recall. While certainly speculative, given that both groups received identical discussion regarding agency duties in lecture (and thus in a decontextualized, abstract sense), the fact that the DMDGC group was able to better appreciate this kind of information seems to again suggest that the concrete experience helped make this understanding of professional duties more accessible to these learners.
As such, this overall pattern of results suggests that learners were gaining a more complete “T-shaped profile” of hydrological education (Ruddell and Wagener, 2013), balancing an increase in not only their specialized conceptual, quantitative, and modeling skills within the field but also achieving a more broad understanding of the role of professionals in the field relative to real-world scenarios. This is a very encouraging result, as it suggests a dual benefit for such DMDGC training.
Further, another interesting point is that it is likely such multiple effects were observed because the participants in this study were just beginning their education in the field of hydrology, so issues of contextualized knowledge or self-selection were likely minimized in this sample. In other words, because learners were lacking a well-defined representation of not only the knowledge of the field but also the role of working professionals in the field, this training experience permitted them to gain greater insight into both the field and requisite application. This fact is even more encouraging as it suggests that such interventions, introduced early in the educational trajectory, can provide a more robust and complete learning experience at all levels. It is possible that such increases in depth and breadth of knowledge early on could translate into more success with the material, thus likely increasing the likelihood of learners persisting in the pursuit of education in this domain. It appears that working with authentic data increases the appreciation of a novice student for the importance of the hydrology profession and for the physical problems this profession addresses.
For the fundamentally place-based geosciences such as hydrology, the integration of concepts will inevitably require exposure to real-world contexts and data. The results of this study demonstrate that computerized learning content can effectively bring the “real world” into the classroom and make it accessible, especially in the case of students at lower levels and across the general curriculum. The findings of this paper also indicate that it is possible to deliver this type of content in a localized place-based context, and to realize learning gains on both physical and professional learning outcomes without introducing a great deal of complexity in the way of computer modeling and programming. A simple spreadsheet, combined with readily available online hydrological data, is sufficient in this case. In other words, these computerized techniques afford instructors the opportunity to have their students engage in realistic and authentic problem-based activities without the need to manage other logistical constraints often encountered with field research (transportation, materials, etc.). It is our hope that the positive findings of this study encourage investment in development of high-quality DMDGC learning materials, and the wider adoption of place-based DMDGC learning materials across the civil engineering curriculum. Implementing such learning experiences into the curriculum will ideally create more enriching experiences for student learners, and hopefully also develop more well-rounded and skilled practicing hydrologists.
While the current study focused on lower-division students, in future work it would also be of interest to expand this program longitudinally throughout the curriculum to identify how to best deliver DMDGC content at all levels of the hydrology curriculum to maximize its effectiveness. Efforts are currently ongoing to do exactly this, and also expand the application of DMDGC content to hydrological concepts beyond flooding and urbanization.
This material is based upon work supported by the National Science Foundation under Grant no. 1043996. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Publication of this article in an open access journal was funded by the Oregon State University Libraries and Press Open Access Fund. Edited by: S. Uhlenbrook