Modeling
All models are wrong, some are useful (Box 1976)
I’ve found it useful to frame the act of modeling as the step where we take the subjective scales we’ve chosen and the scientific method to create a usable piece of evidence for sensemaking.
Model outputs are a function of the model parameters, boundary conditions, the processes represented in the model, and the numerical implementation of those processes. Model intercompairisons are ostensibly aimed at teasing those factors apart but efforts to do so are clouded by the choice of accuracy metric, implicit user value judgment, and uncertainly. Accurately accounting for and isolating the effects of those different pieces is rarely as straightforward as it’s reported to be, and the technical and methodological skill needed to accomplish that is quite rare.
The Modeling-Scale Feedback Loop
Model outputs are a direct function of the model parameters, boundary conditions, and the processes we choose to represent. However, these choices are rarely objective; they are constrained by the scale of the problem we are trying to solve and the most convenient tools and information at hand. When we model, we are making an implicit argument about what problem we are trying to solve, and which scales matter (and consequently, what can be ignored). If we don’t align these with the geographic context of the problem, we aren’t creating a “useful” model; we’re creating a distraction.
Bridging the Observation Gap
We suffer from a massive observation gap; we have limited snapshots of the world but want to understand the whole picture. Modeling can be our attempt to fill those gaps. The larger danger in that arises when we discard primary observations in favor of more cost effective digital representations. At this point, we’re constructing models of convenient pictures, not reality. To prevent this, our models must be more than just code; they must be part of a reproducible toolset. We also need to reemphasize that the scientific method leads with observation, that observing the world is critical to a successful outcome, that primary observations are not as expensive to collect as they are treated, and that this much time in front of a computer is just not good for us.
The Implementation Crisis
The act of modeling has always been a demonstration of competence but that demonstration has taken on new forms as our tools evolve and we gain skills to reproducably and accessibly communicate findings to the different actors across our problem space. It’s also taken on new dimensions, as the technological and societal facets of our wicked problem become more entangled and the expectation of rapid, definitive and defensible answers increases. That “data driven” answer is the standard to which that should be held, otherwise the model is not a scientific instrument; it’s a spatially specific opinion. As someone whose goal is to be helpful to my community and the American public, I hold these charges in high regard and hope that, through these efforts, I’ve provided a glimpse at the sort of reproducible explainability I’m searching for as I attempt to become a competent modeler.
From Model to Map
Finally, modeling is rarely the end of a workflow, but the transition from model space to map space is rarely given the consideration needed to separate out the concerns of the modeler from that of the decision maker. It takes competence to wear the many different hats that have sway in that decision, and a common failure point is that it becomes overwhelming to ensure that the uncertainty inherent in the modeling process don’t disappear when the map is used to make a decision, but that choice is made with the confidence of the process that built it.