OWP FIM

udunits database from /usr/share/xml/udunits/udunits2.xml
This build of rgl does not include OpenGL functions.  Use
 rglwidget() to display results, e.g. via options(rgl.printRglwidget = TRUE).

From: https://github.com/NOAA-OWP/inundation-mapping/wiki/3.-HAND-Methodology

Executive summary

The generation of flood inundation maps is expensive, technically demanding, and heavily dependent on geography and user specified inputs. With the advances of the National Water Model and geospatial approximations to the hydraulic sciences, the ability to generate flood inundation maps via a Height Above Nearest Drainage and Synthetic Rating Curve library was realized. This technique is used in NOAA Office of Water Prediction operational flood forecasting as the FIM4 HAND libraries sheparded by the National Water Center. See the repo and accompanying wiki for more details.

Intro

As one of the first methods of tying the National Water Model to a flood map, the once NFIE then CFIM > FIM2 > Cahaba (FIM3) > GMS (FIM4) codebase is the Office of Water Predictions default means of creating FIM libraries. This current iteration of the OWP HAND based FIM repo is written in Python, distributed as a docker’d environment, and is executed in AWS cloud services. This repo performs the steps needed to create a Hight Above Nearest Drainage (HAND) Relative Elevation model (REM) and Synthetic Rating Curve (SRC). As both a research and an operational toolchain, this repo furthermore serves as an accessible entry point to some of the more “modern” computational patterns and better practices, and can be deployed in a number of ways.

Quickstart for advanced users

The most relevant stearing files are the /src/bash_variables.env You enable HUCS with an inclusion in a list a la /data/inputs_huc_lists/testunit.lst

While there are “smart defaults” for many of the runtime arguments, this tool is designed primarily for use at OWP and with the goal of generating outputs for the NWS at a continental scale. If you are not that use case, you may need to tweak the inputs intelligently and remember to update config\params_template.env to reflect the data you have available in your environment.

You can look for help with the -h flag: /foss_fim/fim_pipeline.sh -h

Did that not make sense? That’s O.K! See below for more verbose instructions and relevent background.

Resources

https://github.com/NOAA-OWP/inundation-mapping

Tutorials

Explainers

What is FIM4 doing?

The process of creating the FIM library is a complex one. See this repo workflow broken down in visual form here (also note the beautiful graphic from the wiki; pilfered and pasted at the top of this page as well!).

How-To’s

How to “run” the repo from pull to HAND.tif

See How I install and use docker for tips on setting up the environment

Installation

Gather Input data

Grab input data.

```{bash}
# aws config
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/3dep_dems/10m_5070 . --exclude '*' --include 'HUC6_1209**_dem.tif' --request-payer requester --dryrun
aws s3 cp s3://noaa-nws-owp-fim/hand_fim/inputs/3dep_dems/10m_5070/fim_seamless_3dep_dem_10m_5070.vrt . --request-payer requester --dryrun
aws s3 cp s3://noaa-nws-owp-fim/hand_fim/inputs/3dep_dems/10m_5070/HUC6_dem_domain.gpkg . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/ahps_sites . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/bathymetery . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/huc_lists . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/landsea . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/nhdplus_vectors_aggregate . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/nld_vectors . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/nwm_hydrofabric . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/rating_curve . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/usgs_gages . --request-payer requester --dryrun
aws s3 cp --recursive s3://noaa-nws-owp-fim/hand_fim/inputs/wbd . --request-payer requester --dryrun
```

Stage inputs

The pre_clip step shaves lots of time off the standard pattern of file > open.

Build the docker image

  1. Open admin powershell
```{powershell}
wsl
```

Get an error? Maybe start docker?

```{sh}
systemctl start docker
```
  1. Pull the repo and build the Image
```{sh}
> cd /mnt/OWP_FIM/inundation-mapping/
git clone https://github.com/NOAA-OWP/inundation-mapping.git
cd ./inundation-mapping/
docker build -f Dockerfile.dev -t owpfim:1.0 /mnt/OWP_FIM/inundation-mapping/inundation-mapping
```

See the changelog for why .dev is now included

create the container

```{powershell}
docker run \
    --rm \
    -it \
    --name 'FIM4' \
    -v /.../root/projects/floodmapping/methods/OWP_FIM/inundation-mapping/inundation-mapping/:/foss_fim \
    -v /.../root/projects/floodmapping/methods/OWP_FIM/outputs/:/outputs \
    -v /.../data/raw/FIM4/:/data \
    owpfim:1.1
```

Run pipeline

The most relevant steering files are the /src/bash_variables.env You enable HUCS with an inclusion in a list a la /data/inputs_huc_lists/testunit.lst

While there are “smart defaults” for many of the runtime arguments, this tool is designed primarily for use at OWP and with the goal of generating outputs for the NWS at a continental scale. If you are not that use case, you may need to tweak the inputs intelligently and remember to update config\params_template.env to reflect the data you have available in your environment.

You can always look for help with the -h flag: /foss_fim/fim_pipeline.sh -h

```{sh}
fim_pipeline.sh -u '12090301' -n FIM4 -ud NONE -bd NONE -zd NONE -skipcal
```

Copy files into data path

[[20220908105839]]

Make input files

```{sh}
python3 /foss_fim/data/wbd/generate_pre_clip_fim_huc8.py -n /data/inputs/pre_clip_huc8/20240927 -u /data/inputs/huc_lists/testunit.lst

#python3 /foss_fim/data/aggregate_vector_inputs.py
python3 /foss_fim/src/preprocess_rasters.py -dem_dir /data/inputs/nhdplus_rasters/
```
```{sh}
python3 /foss_fim/data/acquire_and_preprocess_inputs.py -u 1209 <huc4s_to_process>
#python3 /foss_fim/data/aggregate_vector_inputs.py
python3 /foss_fim/src/preprocess_rasters.py -dem_dir /data/inputs/nhdplus_rasters/
```

python3 /foss_fim/data/acquire_and_preprocess_inputs.py -u 0804 python3 /foss_fim/data/preprocess_rasters.py -dem_dir /data/inputs/nhdplus_rasters/

https://github.com/NOAA-OWP/inundation-mapping/blob/dev/data/bridges/pull_osm_bridges.py#L321 https://github.com/NOAA-OWP/inundation-mapping/blob/dev/data/wbd/generate_pre_clip_fim_huc8.py#L136

Run pipeline

```{sh}
fim_pipeline.sh -u '12090301' -n raw_env -ud NONE -bd NONE -zd NONE -skipcal
```