Merge pull request ProjectPythia#14 from NicholasCote/main

DRAFT: Project Pythia 2024 Cook Off: Interactive ERA5 Dashboard [ DO NOT MERGE]

.isort.cfg

@@ -1,2 +1,2 @@
 [settings]
-known_third_party =
+known_third_party =holoviews,hvplot,panel,xarray

Containerfile

@@ -0,0 +1,27 @@
+# Use an official Python runtime as a base image
+FROM docker.io/mambaorg/micromamba:latest
+
+USER root
+
+RUN apt-get update && apt-get install -y git-all
+
+# Set the working directory in the container to /app
+WORKDIR /home/mambauser/app
+
+# Copy the current directory contents into the container at /usr/src/app
+RUN git clone https://github.com/NicholasCote/ERA5_interactive-cookbook-ncote.git
+
+# Install any needed packages specified in requirements.yml
+RUN micromamba env create -f ERA5_interactive-cookbook-ncote/environment.yml
+
+RUN mv ERA5_interactive-cookbook-ncote/notebooks/04_dashboard.ipynb .
+
+RUN rm -r ERA5_interactive-cookbook-ncote/
+
+# Activate the environment by providing ENV_NAME as an environment variable at runtime 
+# Make port bokeh application port to the world outside this container
+EXPOSE 5006
+
+USER mambauser
+
+CMD ["panel", "serve", "04_dashboard.ipynb", "--allow-websocket-origin=*", "--autoreload"]

README.md

@@ -33,7 +33,7 @@ In the notebooks which comprise this Cookbook, we demonstrate the following:
 
 ## Authors
 
-[Kevin Tyle](https://github.com/ktyle), [Michael Barletta](https://github.com/Michael-Barletta)
+[Kevin Tyle](https://github.com/ktyle), [Michael Barletta](https://github.com/Michael-Barletta), [Negin Sobhani](https://github.com/negin513), [Nicholas Cote](https://github.com/ncote) , [Harsha Hampapura](https://github.com/hrhampapura) , and [Philip Chmielowiec](https://github.com/philip2c)
 
 We also gratefully acknowledge the Google Cloud Research team for making an ARCO-friendly version of the ERA-5 available. Citations for their effort and the ERA-5 reanalysis are below:
 
@@ -65,18 +65,32 @@ We also gratefully acknowledge the Google Cloud Research team for making an ARCO
 
 ## Structure
 
-This cookbook currently consists of two notebooks that access, regrid, and visualize the ARCO ERA-5 repository.
+This cookbook currently consists of multiple  notebooks that access, regrid, and visualize the ARCO ERA-5 repository. Additionally we cover a section on how to preprocess and create ARCO files. 
 
 Additional notebooks will follow.
 
 ### Section 1 ( "Basic Visualization" )
 
 This notebook reads in a sea-level pressure ERA-5 grid, regrids from Gaussian to Cartesian coordinates, and visualizes the data with Matplotlib and Cartopy.
 
-### Section 2 ( "Interactive Visualization 1" )
+### Section 2 ( "Interactive Visualization Part 1: `GeoViews`" )
 
 This notebook reads in sea-level pressure and 2-meter temperature ERA-5 grids, regrids as in the first notebook, and visualizes the data in an interactive manner using [Geoviews](https://geoviews.org).
 
+### Section 3 ("Interactive Visualization Part 2: `hvPlot`")
+This notebook reads in annual average 2-m temperature from RDA's Zarr store and visualizes the data using `hvPlot`. The notebook also demonstrates how to create a simple interactive plot that allows the user to select a specific year and visualize the 2-m and how to create animations.
+
+### Section 4 ("Creating an Interactive Dashboard with `Panel`")
+This notebook demonstrates how to create an interactive dashboard using `Panel` that allows the user to select a specific year and visualize the 2-m temperature. 
+
+## Preprocessing Notebooks for NCAR RDA
+### Section 5 ( "Generate annual/yearly Zarr stores from hourly ERA5 NetCDF files on NCAR’s Research Data Archive")
+This notebook demonstrates how to preprocess hourly ERA5 NetCDF files from NCAR's Research Data Archive (RDA) and generate annual/yearly Zarr stores.
+
+### Section 6 ( "Calculate Temperature Anomalies")
+This notebook demonstrates how to calculate temperature anomalies from the annual 2-m temperature Zarr store generated in Section 5.
+
+
 ## Running the Notebooks
 
 You can either run the notebook using [Binder](https://binder.projectpythia.org/) or on your local machine.

_config.yml

@@ -2,13 +2,16 @@
 # Learn more at https://jupyterbook.org/customize/config.html
 
 title: ARCO ERA-5 Interactive Visualization
-author: Michael Barletta and Kevin Tyle
+author: Michael Barletta, Kevin Tyle, Negin Sobhani, Nicholas Cote, Harshah Hampapura, Philip Chmielowiec 
 logo: notebooks/images/logos/pythia_logo-white-rtext.svg
 copyright: "2024"
 
 execute:
   # To execute notebooks via a Binder instead, replace 'cache' with 'binder'
   execute_notebooks: cache
+  exclude_patterns:
+    - '*era5_anomaly*'
+    - '*data_preprocessing*'
   timeout: 600
   allow_errors: False # cells with expected failures must set the `raises-exception` cell tag
 
@@ -28,7 +31,7 @@ parse:
 sphinx:
   config:
     linkcheck_ignore: ["https://doi.org/*", "https://zenodo.org/badge/*"] # don't run link checker on DOI links since they are immutable
-    nb_execution_raise_on_error: true # raise exception in build if there are notebook errors (this flag is ignored if building on binder)
+    nb_execution_raise_on_error: false # raise exception in build if there are notebook errors (this flag is ignored if building on binder)
     html_favicon: notebooks/images/icons/favicon.ico
     html_last_updated_fmt: "%-d %B %Y"
     html_theme: sphinx_pythia_theme

_toc.yml

@@ -4,12 +4,14 @@ parts:
   - caption: Preamble
     chapters:
       - file: notebooks/how-to-cite
-  - caption: Visualization notebooks
+  - caption: Visualization Notebooks
     chapters:
       - file: notebooks/01BasicVisualization
       - file: notebooks/02InteractiveVisualization
-      - file: notebooks/03_intro_to_interactive_viz.ipynb
+      - file: notebooks/03_hvplot
+      - file: notebooks/04_dashboard
 
   - caption: Preprocessing Notebooks for NCAR RDA
     chapters:
-      - file: notebooks/01_data_preprocessing.ipynb
+      - file: notebooks/05_data_preprocessing
+      - file: notebooks/06_era5_anomaly

environment.yml

@@ -22,3 +22,6 @@ dependencies:
   - gcsfs
   - cf_xarray
   - sphinx-pythia-theme
+  - hvplot
+  - spatialpandas
+  - watchfiles

notebooks/01BasicVisualization.ipynb

@@ -11,19 +11,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# 01_BasicVisualization"
+    "# Basic Visualization using `matplotlib` and `Cartopy`"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": []
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": []
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -6291,7 +6281,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3.9.6 64-bit",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -6305,7 +6295,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.6"
+   "version": "3.10.13"
   },
   "nbdime-conflicts": {
    "local_diff": [

notebooks/02InteractiveVisualization.ipynb

@@ -11,7 +11,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# 02_InteractiveVisualization Part 1: Geoviews"
+    "# Interactive Visualization using `GeoViews`"
    ]
   },
   {
@@ -536,7 +536,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3.9.6 64-bit",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -550,7 +550,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.6"
+   "version": "3.10.13"
   },
   "nbdime-conflicts": {
    "local_diff": [