On 28^th October, 2020 GMDSI hosted a discussional webinar whose purpose was to address tensions which sometimes arise between modellers and hydrogeologists

If you would like to watch the webinar, click here.

After the webinar, some of our listeners had their say – you can read their comments or contribute your own at the bottom of the page. 

John Doherty and Catherine Moore (members of the GMDSI project team) have summarized the discussion, and proposed a few ideas from the GMDSI perspective: this summary is as follows. 

General

GMDSI hosts a number of webinars each year. These are of two types, namely:

1. instructional webinars; and
2. discussional webinars.

Instructional webinars are delivered by one or a number of experts in a particular aspect of decision-support modelling. Their aim is to disseminate knowledge. Selection of the webinar topic is based on the premise that it will interest a significant number of people in the groundwater industry.

Discussional webinars are different. They are normally delivered by a number of experienced people that comprise a panel. This panel discuss an issue that may be of interest to a broad cross section of the groundwater community. The issue may be controversial, or one that is often “swept under the rug” in formal professional discourse. Following presentations by members of the panel, the audience can participate in the discussion. The conversation can then continue in “discussion group” format on GMDSI web pages. Finally, GMDSI personnel author a document which summarizes the discussion, and contributes their own point of view. This is such a document.

Hydrogeologists and Modellers

On 28^th October, 2020 GMDSI hosted a discussional webinar whose purpose was to address tensions which sometimes arise between modellers and hydrogeologists. We asked four experienced and respected members of the groundwater community to comprise our panel. The complexion of the webinar was intended to be light-hearted. We asked our presenters to express any frustrations they may have experienced in a good humoured and constructive way. This, we hoped, would allow listeners to hear some points of view which they may be otherwise unaccustomed to hearing, but which may resonate with them.

Our four speakers were (in order):

• Wendy Timms, professor of environmental engineering at Deakin University;
• Chris Nicol, hydrogeologist at Groundwater Logic Pty, Ltd;
• Rick Evans, principal hydrogeologist at Jacobs;
• Hugh Middlemis, principal engineer at HydroGeoLogic Pty, Ltd.

None of our speakers would classify themselves as either “a hydrogeologist” or as “a modeller”, as their skillsets are broader than this. However Chris and Hugh have devoted much of their careers to modelling, while Wendy and Rick have spent the greater part of their time in site characterization, data acquisition and conceptual model development.

A brief summary of the main points made by each of these speakers follows. We apologize if we have misrepresented them in any way.

Summary of Presentations

Wendy Timms

Wendy stressed the importance of gathering scientific data, processing that data in effective ways, and of ensuring the integrity of its expression in a model. Information forthcoming from data is expressed through the conceptual model which underpins the numerical model. This includes information on system hydraulic properties that are represented by model parameters. Values for parameters may be assigned directly, and/or they may be estimated through history-matching against appropriately processed field measurements.

Wendy noted that even after assimilation of all available data into a model, the uncertainties associated with decision-critical model predictions may be high. These should be quantified so that decision-makers are aware of this. She also noted that modellers should be prepared to consider the use of all kinds of data, including those that are sometimes overlooked but whose information content may still be considerable. The latter include tracers, estimates of groundwater age, and passive estimates of aquifer storativity. The modelling process itself should also be queried in order to suggest data locations and types that can reduce the uncertainties of management-salient predictions.

Chris Nicol

Chris reiterated the importance of data assimilation and uncertainty quantification in decision-support modelling. He noted that this places certain requirements on a model. In particular, a decision-support model should be fast and lightweight. This can be achieved by dispensing with non-essential complexity. However Chris also pointed out that modelling stakeholders may disagree in their assessment of what is essential and what is non-essential in a particular model. He demonstrated how disagreements on this issue can sometimes be resolved using linear analysis. In particular, if a contentious aspect of a model’s construction is appropriately parameterized, its contribution to the uncertainties of decision-critical predictions can be assessed. If this contribution is small compared with uncertainties induced by other aspects of a model’s construction and/or parameterisation, then consideration can be given to representing it in a simplistic or abstract manner if this increases model execution speed, and/or improves the model’s numerical stability.

Rick Evans

Rick reminded the webinar audience that in any particular modelling context a third party is involved in negotiations of what a model should look like, and what role it should perform. This is the client. Often, a client has little interest in how complex or simple a model should be, or how well its outputs should match measurements of system behaviour. He/she is even less interested in uncertainty quantification. His/her only concern is to acquire the information which he/she needs in order to make a decision. Rick pointed out that a hydrogeologist must often arbitrate between a modeller who seeks integrity of the modelling process (however this is defined) and a client who wants a cheap solution to a pressing problem.

While acknowledging that model predictions are always uncertain, Rick questioned the need for rigorous uncertainty analysis on every occasion of model deployment. He suggested that on many occasions a simple model (often an analytical model), together with “professional judgment”, provides adequate decision support. He asserted that modellers can sometimes become rather obsessed with their technology as it pertains to simulation, history-matching and/or uncertainty analysis while losing sight of the larger decision-support picture. An “informed hydrogeologist” should ensure that modellers maintain allegiance to an underlying hydrogeological conceptual model on the one hand, and to the decision-support needs of the client on the other hand.

Hugh Middlemis

Hugh reminded his audience that modellers and hydrogeologists are generally part of a larger team that includes other disciplines such as engineering, geophysics and ecology. Personally, he has experienced little tension between members of such multidisciplinary teams. However, tensions are sometimes introduced by third parties, for example by client or stakeholder groups for whom modelling has raised issues that had not been hitherto considered. Hugh pointed out that it is not uncommon for a group that is opposed to a proposed project, or for regulators that must operate in contentious situations, to request a high level of model complexity. It is Hugh’s opinion, however, that the level of modelling effort should be commensurate with the risks associated with decisions that modelling is intended to support. Thus it is not hydrogeological considerations alone which should determine the design of a model, for decision-support and social contexts should also play a role. Hugh also pointed out that complex models are not necessarily better models, for the introduction of complexity to a model may unwittingly instil bias in some of its predictions.

The Authors’ Perspective

Thoughts on the Talks Themselves

It is our view that all of the speakers made good points. (What else can one expect from those with many decades of collective experience?)

The tone of all of the talks was conciliatory. Speakers emphasized the positive aspects of any creative tension that may exist between hydrogeologists and modellers. Frustrations that a hydrogeologist may experience if he/she considers that a model is too abstract, or that a modeller may experience if he/she is asked to build a “geologically realistic” model that is too unwieldy to support history matching or uncertainty analysis were mentioned, but were not emphasized. The decision-support utility of simple models was acknowledged by all speakers, as were the pitfalls of excessive model complexity.

Our Own Reflections

We agree with the panel that groundwater modelling is a compromise. This compromise is an outcome of tensions between:

• representation of hydrogeological detail, or more abstract representation of broadscale hydrogeological features, in a numerical model;
• attainment of a good fit between model outcomes and field measurements of system behaviour, or sacrificing that fit so that a numerical model can remain strategically simple;
• attainment of a good fit between model outcomes and field measurements, or sacrificing that fit so that a numerical model can be complex – too complex for efficient history-matching;
• formal quantification of predictive uncertainty, or association of a safety margin with model predictions based on informed judgement.

The panel made it clear that those with experience in the groundwater industry are comfortable with these tensions; they attempt to resolve them in their own ways according to the needs of a particular job. However they also pointed out that client, stakeholder and government groups may not be so comfortable with these tensions. The expectations of these groups may exceed what modelling can deliver, or may distract modelling from the best way that it can fulfil its decision-support potential. Part of the job description of both modellers and hydrogeologists is to challenge these expectations. Chris demonstrated the use of mathematical tools that can formalize the search for an optimal compromise between model “realism” and strategic abstraction; Rick and Hugh emphasized the need for continued industry dialogue on this issue – a dialogue to which they have both contributed (see references below).

We, the authors of this document, also recognize this tension. Furthermore, we recognize the need for continued and fluid discussions on it. This, indeed, is part of the mandate of GMDSI.

Many model design decisions are necessarily subjective. However subjectivity must be guided by logic. So some rules apply. When a model is used to support groundwater management, it must be acknowledged that any particular prediction made by that model is probably wrong. Assessment of the potential for predictive wrongness (i.e. of predictive uncertainty) must therefore accompany the making of predictions by a model. Furthermore, this potential must not be understated. After all, the reason for modelling is to prevent surprises. At the same time, depending on the management context, attempts should be made to reduce this uncertainty by tapping the information content of available data. It is to these principles that subjective model design decisions must be referred.

If uncertainties are high because data is scarce, a model can be simple, as Rick pointed out. The ability of such a simple model to reproduce field measurements may be limited. However if field measurements are few, little is lost. The ability of such a model to quantify predictive uncertainty may also be limited. However, a simple model can be readily deployed for worst case analysis. Alternatively, or as well, an uncertainty interval based on “professional judgement” can be bestowed on its predictions. Obviously, stakeholders must be confident that this interval covers the true range of predictive possibilities.

As Chris pointed out, a properly-designed, fast-running model of medium complexity is often capable of quantifying uncertainties associated with its predictions. Where data are available, it can reduce these uncertainties through history-matching. However, as Chris also pointed out, “fast-running” comes at a cost. Many aspects of its construction may be abstract rather than “picture perfect”. This may limit the ability of a hydrogeological conceptual model, and/or direct field measurements of system properties to inform them. A significant benefit of physically-based models (to which Wendy drew our attention) is thereby eroded. Furthermore, theory shows that estimation of inappropriately abstract model parameters through history-matching may induce bias in some predictions. Fortunately, linear analysis exemplified by Chris may identify these predictions, and the extent to which they may undergo simplification-induced bias.

The authors of this document agree with the panel on the ever-present pressure to build an unnecessarily complex model. We also agree that, more often than not, this pressure comes from stakeholder groups whose knowledge of groundwater processes is limited, rather than from either modellers or hydrogeologists. (However, it must be acknowledged that sometimes this pressure can come from modellers and hydrogeologists as well.)

It would be interesting to inquire into the mindset which induces non-experts (and some experts) to request such a model. Perhaps this is a webinar topic for another day. Misapprehensions which prompt such requests probably arise from the meaning of “model” in other spheres of life. Most (but not all) groundwater professionals are aware that a complex numerical groundwater model runs slowly and is held constant hostage to numerically instability. Meanwhile its inclusion of details whose properties and dimensions can only be guessed requires stochastic representation of these details.  Attempts to constrain the stochastic representation of “realistic” detail embodied in complex models, such that model outputs match field measurements of system behaviour is a next-to-impossible numerical undertaking.

The authors feel that the issues that were exposed by this webinar place at the feet of our industry’s research institutions a set of tasks that require their urgent attention. Decision support modelling requires many context-specific compromises. However, while the rules of logic apply, application of these rules in different modelling contexts is difficult. A particular modelling strategy may make a model’s parameters more recognisable by hydrogeologists who can then inform them through field studies. Or it may make its parameters more abstract, but more amenable to adjustment so that they can be informed by measurements of system state. Something is gained, and something is lost through either choice. We, as an industry, must develop the skills that are required to quantify these gains and losses in the variety of decision-support contexts that we face in our day-to-day work. If the search for an optimal compromise in any decision-support context has scientific justification, we can all agree on it. Once we agree on it, then we as an industry can convince the rest of society that numerical simulation is indeed protecting its interests.

In summary, we need to be able to assure a sceptical public that:

• Replication in a numerical groundwater model of all nuances of subsurface water and solute movement is both unnecessary and beyond our capabilities.
• An imperfect replication of the real world does not prevent us from harvesting information from environmental data. This information can be put to good use through numerical simulation in order to ensure that no awkward surprises accompany the choice of a particular course of management action.
• The second of these should not be confused with the first.

Some References

Barnett, B., Townley, L.R., Post, V., Evans, R.E., Hunt, R.J., Peeters, L., Richardson, S., Werner, A.D., Knapton, A and Boronkay, A., 2012. Australian Groundwater Modelling Guidelines. Waterlines Report, National Water Commission, Canberra.

Doherty, J., 2011. Modeling: picture perfect or abstract art? Ground Water, 49(4), 455-456.

Doherty, J. and Moore, C., 2019. Decision Support Modeling: Data Assimilation, Uncertainty Quantification and Strategic Abstraction. Groundwater 58(3) 327-337 doi: 10.1111/gwat.12969.

Middlemis, H., Merrick, N., Rozlapa, K. and Ross, J., 2001. Groundwater Flow Modelling Guideline. Murray-Darling Basin Commission.

Middlemis, H. and Peeters L.J.M., 2018 Uncertainty analysis—guidance for groundwater modelling within a risk management framework. A report prepared for the Independent Expert Scientific Committee on Coal Seam Gas and Large Coal Mining Development through the Department of the Environment and Energy, Commonwealth of Australia, 2018.