FIM validations: Mosaics of catfim
There are lots of ways we map FIM. Let’s take a look at some of the most popular, a gage based FIM library, a HAND library, and a categorical FLIDPLN library. We’ll use the gage fim as a ground truth. Let’s test this with the MHKK1 site. This ground truth FIM looks a little something like so:
FLDPLN is a stage only prediction
HAND predicts both an elevation and a rating curve per forecast feature (as opposed to the global access pattern that is used for the categorical FIM’s)
Using that gage library as a “ground truth”, let’s look at the accuracy of our other libraries.
(and their shared extent)
To compare these, we’ll deploy the following common metrics:
Flood Model Performance Metrics
| Metric | Range | Ideal | Description | Hydrologic Context |
|---|---|---|---|---|
| CSI | 0 to 1 | 1 | \frac{TP}{TP + FP + FN} | Benchmarks for flash flood guidance often peak around 0.20 to 0.44; global models frequently score between 0.4 and 0.6. |
| F1 | 0 to 1 | 1 | \frac{2TP}{2TP + FP + FN} | Usually higher than CSI. High-performing damage prediction models often achieve 0.76 or higher. |
Results
Optimized Stage-Matching: An Apples-to-Apples Comparison
Comparing inundation extents strictly by matching nominal stage values (e.g., 15 ft AHPS vs. 15 ft HAND) evaluates bulk library accuracy. However, a more intentional mapping would use the best available library to match the desired conditions, bypassing conflicts related to differing underlying topographies, channel datum definitions, and hydrologic routing assumptions that are introduced in traditional accuracy assessments. To evaluate the inherent structural capability of the FLDPLN and HAND libraries to capture the spatial pattern of AHPS inundation, we must isolate these factors. By identifying the equivalent stages that ought to be used to access the libraries, we can quantify the systematic vertical bias between the them. A consistent offset may indicate that datum or thalweg alignment issue are present, and a varying offset suggests differing flow-routing physics across the access pattern range.
