Determine which diagnostics to include for Land component

[TeaganKing]

Provide diagnostics from the land component to be included in #3

[TeaganKing]

Assigning to Will now. We will also assign Sam Levis once he accepts the repository invite.

[wwieder]

Here's an example of what our existing diagnostics package generates. Basically

• Global time series & tables
• Global maps (seasonal and annual)
• Regional climatologies
• Regional time series
• Select runoff plots from RTM or MOSART

[wwieder]

On top of this is would be great if we could also feed model output into ILAMB, which hopefully should be pretty straightforward (famous last words).

[wwieder]

I loaded the CUPiD repo and (finally) built the conda environments.
Seems like there may be some issues with the scratch directories the notebooks are trying to read from (e.g. need /glade/cheyenne/scratch).

Error I'm getting is below, @mnlevy1981.

PermissionError: [Errno 13] Permission denied: '/glade/scratch'

ploomber.exceptions.TaskBuildError: Error when executing task 'adf_quick_run'. Partially executed notebook available at /glade/u/home/wwieder/CUPiD/examples/adf-mom6/computed_notebooks/adf-quick-run/adf_quick_run.ipynb
ploomber.exceptions.TaskBuildError: Error building task "adf_quick_run"
------------------------------------------------------------------- NotebookRunner: surface -> File('computed_notebook...-run/surface.ipynb') --------------------------------------------------------------------
-------------------------------------------------------------------------- /glade/u/home/wwieder/CUPiD/examples/nblibrary/surface.ipynb --------------------------------------------------------------------------
---------------------------------------------------------------------------
Exception encountered at "In [2]":
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[2], line 10
      8 from ncar_jobqueue import NCARCluster
      9 from dask.distributed import Client
---> 10 from mom6_tools.DiagsCase import DiagsCase
     11 from mom6_tools.m6toolbox import add_global_attrs
     12 from mom6_tools.m6plot import xycompare, xyplot

ModuleNotFoundError: No module named 'mom6_tools'

ploomber.exceptions.TaskBuildError: Error when executing task 'surface'. Partially executed notebook available at /glade/u/home/wwieder/CUPiD/examples/adf-mom6/computed_notebooks/adf-quick-run/surface.ipynb
ploomber.exceptions.TaskBuildError: Error building task "surface"
=============================================================================================== Summary (2 tasks) ================================================================================================
NotebookRunner: adf_quick_run -> File('computed_notebook...df_quick_run.ipynb')
NotebookRunner: surface -> File('computed_notebook...-run/surface.ipynb')
================================================================================================ DAG build failed ================================================================================================

Need help? https://ploomber.io/community

[mnlevy1981]

Are you running this on derecho or casper? I get similar errors on derecho, but can run successfully on casper -- I think I can change /glade/scratch to /glade/cheyenne/scratch everywhere it still appears to get it to work on both machines; I'll give that a try and submit a PR if it works.

[wwieder]

I was trying on derecho, but can try on casper later today or tomorrow.

[wwieder]

casper also failed with this message:

  File "/glade/u/home/wwieder/miniconda3/envs/cupid-dev/lib/python3.11/site-packages/ploomber/executors/parallel.py", line 321, in __call__
    raise DAGBuildError(str(BuildExceptionsCollector(exps)))
ploomber.exceptions.DAGBuildError: 
============================================================================ DAG build failed =============================================================================
------------------------------------------------ NotebookRunner: surface -> File('computed_notebook...-run/surface.ipynb') ------------------------------------------------
------------------------------------------------------ /glade/u/home/wwieder/CUPiD/examples/nblibrary/surface.ipynb -------------------------------------------------------
---------------------------------------------------------------------------
Exception encountered at "In [2]":
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[2], line 10
      8 from ncar_jobqueue import NCARCluster
      9 from dask.distributed import Client
---> 10 from mom6_tools.DiagsCase import DiagsCase
     11 from mom6_tools.m6toolbox import add_global_attrs
     12 from mom6_tools.m6plot import xycompare, xyplot

ModuleNotFoundError: No module named 'mom6_tools'

ploomber.exceptions.TaskBuildError: Error when executing task 'surface'. Partially executed notebook available at /glade/u/home/wwieder/CUPiD/examples/adf-mom6/computed_notebooks/adf-quick-run/surface.ipynb
ploomber.exceptions.TaskBuildError: Error building task "surface"
============================================================================ Summary (1 task) =============================================================================
NotebookRunner: surface -> File('computed_notebook...-run/surface.ipynb')
============================================================================ DAG build failed =============================================================================

Need help? https://ploomber.io/community

[mnlevy1981]

From

ModuleNotFoundError: No module named 'mom6_tools'

it looks like mom6_tools didn't get installed in cupid-analysis. Can you do the following?

$ conda activate cupid-analysis
(cupid-analysis) $ cd ~wwieder/CUPiD/externals/mom6-tools/
(cupid-analysis) $ pip install -e .

[wwieder]

Now it's a data permission issue

PermissionError: [Errno 13] Permission denied: '/glade/campaign/cgd/oce/datasets/cesm/tx2_3/mld/deBoyer2004/deBoyer04_MLD_remapped_to_tx2_3.nc'

ploomber.exceptions.TaskBuildError: Error when executing task 'surface'. Partially executed notebook available at /glade/u/home/wwieder/CUPiD/examples/adf-mom6/computed_notebooks/adf-quick-run/surface.ipynb
ploomber.exceptions.TaskBuildError: Error building task "surface"

[wwieder]

OK, I'm able to run the CUPiD example provided on the README! Next step seems to be to modify some existing workflow to point to some land model output and make plots.

[wwieder]

Here's an example notebook that may be a good starting point for LMWG plots, here focusing on global maps and difference plots (comparing two simulations). Maybe we can work tomorrow on how to best merge something like this into the CUPiD platform?

[megandevlan]

Another metric that's likely worth tracking is the coupling index, to measure the strength of the land-atmosphere coupling. We can compute the terrestrial leg entirely on the land side, which would measure how strongly variations in soil moisture (10cm) impact surface fluxes (sensible or latent would both be valid).

I have a rough notebook that does this here, but I'm working to get a cleaner/more isolated version that takes recent CESM development runs, regrids from SE to FV grid, and computes/plots CI. But it's probably worth a conversation soon on the best way to do all that!