cdo_01_calculate_monthly_anomaly_and_nino34_index_cmip6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Calculate Monthly Anomaly and Nino34 Index Using CDO\n",
    "\n",
    "\n",
    "In this notebook we demonstrate how to calculate monthly anomoly and Nino34 index using CDO:\n",
    "\n",
    "- Look at the information contained in our example file\n",
    "- Visualise the data using ncview \n",
    "- Calculate monthly anomaly\n",
    "- Calculate Nino34 Climate Index\n",
    "\n",
    "This example uses the Coupled Model Intercomparison Project (CMIP6) collections. For more information, please visit the [NCI data catalogue]( https://geonetwork.nci.org.au/geonetwork/srv/eng/catalog.search#/metadata/f5194_5909_8003_9216).\n",
    "\n",
    "---\n",
    "\n",
    "- Authors: NCI Virtual Research Environment Team\n",
    "- Keywords: CMIP, CDO, monthly anomaly, Nino34 Index\n",
    "- Create Date: 2019-Oct; Update Date: 2021-Feb\n",
    "---\n",
    "\n",
    "This notebook is licenced under the [Creative Commons Attribution 4.0 International license](https://creativecommons.org/licenses/by/4.0/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is CDO?\n",
    "\n",
    "CDO stands for \"Climate Data Operators\". CDO is a collection of command line operators to manipulate and analyse Climate and NWP model Data. It support the following data formats: GRIB 1/2, netCDF 3/4, SERVICE, EXTRA and IEG. There are more than 600 operators available. See [CDO's homepage](https://code.mpimet.mpg.de/projects/cdo) for more information about this library.\n",
    "\n",
    "To load the CDO module on the VDI, run:\n",
    "\n",
    "``` $ module load cdo```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List the path of the CMIP6 data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we will be using an example CMIP6 file that includes the near surface temperature from the 20th century all-forcing historical simulation based on the CESM2.0 model of NCAR. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc\r\n"
     ]
    }
   ],
   "source": [
    "! ls /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Have a look at the data file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**CDO Basic usage:** \n",
    "`$ cdo info filename | less`\n",
    "\n",
    "**less** display only one page at a time in the termial. You can move forwards and backwards to see more. Press **q** to quit the view."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![1](images/cdo_anomoly_nino1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can use **ncview** to visualise the data.\n",
    "\n",
    "**Basic usage:**\n",
    "\n",
    "```\n",
    "$ ncview <file>\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ncview 2.1.8 David W. Pierce  8 March 2017\n",
      "http://meteora.ucsd.edu:80/~pierce/ncview_home_page.html\n",
      "Copyright (C) 1993 through 2015, David W. Pierce\n",
      "Ncview comes with ABSOLUTELY NO WARRANTY; for details type `ncview -w'.\n",
      "This is free software licensed under the Gnu General Public License version 3; type `ncview -c' for redistribution details.\n",
      "\n",
      "Note: the coordinates attribute for variable tas is being ignored,\n",
      "since it specifies a variable (time) that has 1 effective dims (an effective dim has a size greater than 1)\n",
      "I am not set up to handle cases with coordinate mapping using anything other than 0 or 2 effective dims\n",
      "X connection to :2 broken (explicit kill or server shutdown).\n"
     ]
    }
   ],
   "source": [
    "!ncview /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: If the cell above freezes, interrupt the kernel ard re-run it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data should look something like this: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![3](images/cdo_anomoly_nino3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's see which years this file includes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the function **showyear** to display all the years in this file\n",
    "\n",
    "**Basic usage:**\n",
    "\n",
    "```\n",
    "$ cdo showyear <file>\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning (cdfInqContents): Coordinates variable time can't be assigned!\n",
      " 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014\n",
      "\u001b[32mcdo    showyear: \u001b[0mProcessed 1 variable over 1980 timesteps [0.26s 36MB].\n"
     ]
    }
   ],
   "source": [
    "!cdo showyear /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![2](images/cdo_anomoly_nino2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate monthly anomaly "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, we are going to calculate the monthly anomaly between 1971 and 2000. We will:\n",
    "\n",
    "1. Choose climatology years 1971-2000 \n",
    "2. Calculate the monthly average during 1971-2000\n",
    "3. Subtract the monthly average from the original file to obtain the difference, i.e., anomaly.\n",
    "\n",
    "We need three functions to implement this procedure:\n",
    "\n",
    " \n",
    "```\n",
    "$ cdo selyear, year_lower, year_upper <input.nc> <output.nc> \n",
    "\n",
    "```\n",
    "\n",
    "``selyear`` selects the years of interest and generates a new file.\n",
    "\n",
    "\n",
    "```\n",
    "$ cdo ymonmean <input.nc> <output.nc> \n",
    "\n",
    "```\n",
    "\n",
    "`ymonmean` calculates the mean monthly values from a multi-year monthly timeseries.\n",
    "\n",
    "\n",
    "```\n",
    "$ cdo ymonsub <input1.nc> <input2.nc> <output.nc> \n",
    "\n",
    "```\n",
    "\n",
    "`ymonsub` calculates the difference between two input files and saves this difference as an output file.\n",
    "\n",
    "However, if we do the three steps above we will generate unnecessary intermediate files. CDO allows for the piping of multiple commands so we can chain together the commands above:\n",
    "\n",
    "```\n",
    "$ cdo ymonsub <input.nc> -ymonmean -selyear,1971,2000 <input2.nc> output.nc\n",
    "\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mcdo(1) ymonmean: \u001b[0mProcess started\n",
      "\u001b[32mcdo(2) selyear: \u001b[0mProcess started\n",
      "Warning (cdfInqContents): Coordinates variable time can't be assigned!\n",
      "Warning (cdfInqContents): Coordinates variable time can't be assigned!\n",
      "\u001b[32mcdo(2) selyear: \u001b[0mProcessed 1 variable over 1980 timesteps.\n",
      "\u001b[32mcdo(1) ymonmean: \u001b[0mProcessed 19906560 values from 1 variable over 360 timesteps.\n",
      "\u001b[32mcdo    ymonsub: \u001b[0mProcessed 663552 values from 2 variables over 1992 timesteps [6.56s 93MB].\n"
     ]
    }
   ],
   "source": [
    "### create output directory if it doesn't already exist\n",
    "import os\n",
    "outdir = './output'\n",
    "if not os.path.exists(outdir):\n",
    "    os.mkdir(outdir)\n",
    "    \n",
    "### run our cdo command\n",
    "! cdo ymonsub /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc -ymonmean -selyear,1971/2000 /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc ./output/tas_Amon_CESM2_historical_r1i1p1f1_gn_197101-200012_anomaly.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's have a look at the monthly anomaly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ncview 2.1.8 David W. Pierce  8 March 2017\n",
      "http://meteora.ucsd.edu:80/~pierce/ncview_home_page.html\n",
      "Copyright (C) 1993 through 2015, David W. Pierce\n",
      "Ncview comes with ABSOLUTELY NO WARRANTY; for details type `ncview -w'.\n",
      "This is free software licensed under the Gnu General Public License version 3; type `ncview -c' for redistribution details.\n",
      "\n",
      "calculating min and maxes for tas...\n",
      "X connection to :2 broken (explicit kill or server shutdown).\n"
     ]
    }
   ],
   "source": [
    "!ncview ./output/tas_Amon_CESM2_historical_r1i1p1f1_gn_197101-200012_anomaly.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![4](images/cdo_anomoly_nino4.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Nino34 Index using monthly anomaly\n",
    "\n",
    "There are several indices used to monitor the tropical Pacific, all of which are based on SST anomalies averaged across a given region.  Usually the anomalies are computed relative to a base period of 30 years.  The Niño 3.4 index and the Oceanic Niño Index (ONI) are the most commonly used indices to define El Niño and La Niña events. See more about [sea surface tempreture indices](https://climatedataguide.ucar.edu/climate-data/nino-sst-indices-nino-12-3-34-4-oni-and-tni)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To calcuate Nino34 Index, we need to choose the Nino34 region with lat/lon bbox = (-170,-120,-5,5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mcdo    sellonlatbox: \u001b[0mProcessed 1 variable over 1980 timesteps [1.41s 51MB].\r\n"
     ]
    }
   ],
   "source": [
    "!cdo sellonlatbox,-170,-120,-5,5 ./output/tas_Amon_CESM2_historical_r1i1p1f1_gn_197101-200012_anomaly.nc ./output/Nino34Region.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then use function **fldavg** to calculate the field average and save the result as the Nino34 index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cdo    fldavg:                        1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 91\u001b[32mcdo    fldavg: \u001b[0mProcessed 1 variable over 1980 timesteps [0.44s 38MB].\n"
     ]
    }
   ],
   "source": [
    "!cdo fldavg ./output/Nino34Region.nc ./output/Nino34Idx.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "\n",
    "In this example, we show how to use CDO command line operators to calculate monthly anomaly and Nino34 Climate Index."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference\n",
    "\n",
    "\n",
    "https://code.mpimet.mpg.de/projects/cdo/embedded/cdo.pdf"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}