Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abstract out ADF's time series generation functionality so other components can use it #30

Closed
TeaganKing opened this issue Dec 7, 2023 · 6 comments · Fixed by #78
Closed

Comments

@TeaganKing
Copy link
Collaborator

TeaganKing commented Dec 7, 2023

We should also be sure that this is not a duplicated effort (eg, with Brian Dobbin's tool).

This could also be an area where we could leverage GeoCAT.

@dabail10
Copy link
Collaborator

The ADF tool just uses ncrcat and is not very flexible. The main things we want are chunking and compression. Should this be part of issue #18 ?

@TeaganKing
Copy link
Collaborator Author

TeaganKing commented Jan 18, 2024

In our discussion during the CUPiD hackathon, with @nusbaume @brianpm @justin-richling @kafitzgerald and myself, we decided we'll want to pull out adf_diag.create_time_series(); this will entail a slight rewriting.
We want to avoid circular dependencies. Let's plan to make a timeseries.py file within CUPiD/cupid. Eventually, if we pull in Brian Dobbins' timeseries repository (if it's actually useful to replace), we can replace the timeseries.py file.

@TeaganKing
Copy link
Collaborator Author

We could merge the ADF taylor diagram with GeoCat Viz taylor diagram.

Taylor_plot_setup, plot_taylor_data, and taylor_plot_finalize could probably be replaced by geocat's function. We probably can't replace gathering data from ADF, selecting variables, subsetting for seasonal averaging, etc.

@TeaganKing
Copy link
Collaborator Author

TeaganKing commented Jan 18, 2024

Overarching CUPiD needs: timeseries, regridding, climatology/time averaging, website generation functionality.

Prioritization:

  1. Website generation move into CUPiD -- This currently uses jinja and should be within CUPiD itself rather than universal outside of CUPiD.
  2. GeoCAT could be really useful for regridding and ADF has more flexible time averaging (eg, 4 month seasonal averages or single).
  3. Put timeseries into CUPiD -- if we are prioritizing this, it would be good to know which components need timeseries vs history files?
  4. Determine if some of the more generalized plotting functions are more useful in GeoCAT viz or in CUPiD . (Within ADF/lib/plotting_functions.py, we may want to pull mask_land_or_ocean into a centralized place.) See plots described in Determine which diagnostics to include for Atmospheric component #6

@TeaganKing
Copy link
Collaborator Author

TeaganKing commented Jan 18, 2024

Moving forward:

  • Have GeoCAT member attend ADF hackathon on a Thursday afternoon or another meeting time -- determine which functions we can make into GeoCAT functions - @kafitzgerald can coordinate with GeoCAT to see who can join.
  • Making top level CUPiD website - @justin-richling and @TeaganKing can collaborate on this.
  • See if timeseries is needed for other components during next CUPiD meeting @TeaganKing will add to agenda
  • @justin-richling - time averaging comparison with ADF and GeoCAT comp

@TeaganKing
Copy link
Collaborator Author

From a discussion with @mnlevy1981 regarding the timeseries note-- we'll eventually want to have all components be able to run on timeseries in order to use smaller model output files, but we should probably first focus on some of the other items from the ESDS event hackathon last week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants