Create geometry simplification pipeline for basins

[aufdenkampe]

The TDX Hydro streamnet and streamreach_basins geometries provided by NGA are at delineated at the scale of the underlying DEM pixels. These full resolution geometries are amazing for explicitly knowing which pixels are in/out for purposes of flow directions and zonal stats.

We decided for our initial implementation for visualization that simplifying those geometries was not necessary due to the tiling approaches used by Model My Watershed. Once tiles are rendered the first time and cached, performance for visualization is very fast, regardless of scale.

However, these full resolution geometries add computational and storage that slow down many of our processes, especially for dissolving basin geometries in the final step of delineating large watersheds.

We need to find the best approach and settings for simplifying these geometries just the right amount.

[aufdenkampe]

Commit 5aea157 demonstrates how to use the topojson library to simplify adjacent polygons without introducing gaps or overlaps.

I used tolerance = 0.0001, where 0.00008 is used for MMW HUC12s.

Results:

In addition, this TopoSimplified file is 2.2x faster to write to compressed parquet, stores as a 2.0x smaller file, and reads 1.3x faster.

• NOTE: this was tested with a very small subset (i.e. the 188 LINK Christina River Basin) and should probably be tested with a larger file.

[aufdenkampe]

I just retested the performance benefits of the TopoSimplification on the full sized TDX_streamreach_basins_mnsi_7020038340_01.parquet file (Eastern USA with 140053 LINKNOs), and got these results compared to the full resolution file:

• 2.9x faster write speed
• 2.0x small file size
• 3.5x faster read speed

HOWEVER, running the toposimplify() took 62m 54s on Anthony's new M3-Max Mac!!! That said, there may be ways to further tweak performance by better leveraging the simplification library