Documentation for umbrella run script (#260)

* Improve docs for runots * More docs * docs update * Remove mamba-version
USGS-CMG · Sep 26, 2024 · 81182d4 · 81182d4
1 parent 882b595
commit 81182d4
Show file tree

Hide file tree

Showing 19 changed files with 102 additions and 451 deletions.
diff --git a/.github/workflows/stglib_test_conda.yml b/.github/workflows/stglib_test_conda.yml
@@ -27,7 +27,7 @@ jobs:
       - run: git lfs checkout
       - uses: conda-incubator/setup-miniconda@v3
         with:
-          mamba-version: "*"
+          # mamba-version: "*"
           activate-environment: test-environment
           environment-file: requirements.yml
           python-version: ${{ matrix.python-version }}

diff --git a/doc/aqd.rst b/doc/aqd.rst
@@ -1,31 +1,6 @@
 Aquadopp (currents)
 *******************
 
-Data will generally be processed using a series of run scripts that use command line arguments.  For AQD currents it's a 2 step process.
-
-Instrument data to raw .cdf
-===========================
-
 First, in AquaPro, export data to text files using the default options.
 
-Convert from text to a raw netCDF file with ``.cdf`` extension using runaqdhdr2cdf.py. This script
-depends on two arguments, the global attribute file and extra configuration information :doc:`configuration files </config>`.
-
-runaqdhdr2cdf.py
-----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.aqdhdr2cdf_parser
-   :prog: runaqdhdr2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into an CF-compliant netCDF file with .nc extension, optionally including :doc:`atmospheric correction </atmos>` of the pressure data.
-
-runaqdcdf2nc.py
----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.aqdcdf2nc_parser
-   :prog: runaqdcdf2nc.py
+Then use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
diff --git a/doc/aqdhr.rst b/doc/aqdhr.rst
@@ -3,31 +3,6 @@ Aquadopp HR
 
 **Aquadopp HR mode data has only seen limited testing. Contributions welcome.**
 
-Data are processed using a series of run scripts that use command line arguments.
-
-Instrument data to raw .cdf
-===========================
-
 First, in AquaPro HR, export data to text files using the default options.
 
-Convert from text to a raw netCDF file with ``.cdf`` extension using runaqdhrhdr2cdf.py. This script
-depends on two arguments, the global attribute file and extra configuration information :doc:`configuration files </config>`.
-
-runaqdhrhdr2cdf.py
-------------------
-
-.. argparse::
-   :ref: stglib.core.cmd.aqdhdr2cdf_parser
-   :prog: runaqdhrhdr2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into an CF-compliant netCDF file with .nc extension, optionally including :doc:`atmospheric correction </atmos>` of the pressure data.
-
-runaqdhrcdf2nc.py
------------------
-
-.. argparse::
-   :ref: stglib.core.cmd.aqdcdf2nc_parser
-   :prog: runaqdhrcdf2nc.py
+Then use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
diff --git a/doc/eofe.rst b/doc/eofe.rst
@@ -1,29 +1,4 @@
 EofE ECHOLOGGER
 ***************
 
-Data will generally be processed using a series of run scripts. The first script for each instrument type
-depends on two :doc:`configuration files </config>`.
-
-Instrument data to raw .cdf
-===========================
-
-Convert from  tab or space-delimited log file to a raw netCDF file with .cdf extension using ``runeofelog2cdf.py``.
-
-runeofelog2cdf.py
------------------
-
-.. argparse::
-   :ref: stglib.core.cmd.eofelog2cdf_parser
-   :prog: runeofelog2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into CF-compliant netCDF file with .nc extension using ``runeofecdf2nc.py``.
-
-runeofecdf2nc.py
-----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.eofecdf2nc_parser
-   :prog: runeofecdf2nc.py
+Starting from the tab or space-delimited log file, use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
diff --git a/doc/exo.rst b/doc/exo.rst
@@ -1,29 +1,4 @@
 YSI EXO
 *******
 
-Data will generally be processed using a series of run scripts. The first script for each instrument type
-depends on two :doc:`configuration files </config>`.
-
-Instrument data to raw .cdf
-===========================
-
-Convert from exported .csv file from KOR software to a raw netCDF file with .cdf extension using ``runexocsv2cdf.py``.
-
-runexocsv2cdf.py
-----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.exocsv2cdf_parser
-   :prog: runexocsv2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into an CF-compliant netCDF file with .nc extension using ``runexocdf2nc.py``, optionally including :doc:`atmospheric correction </atmos>` of the pressure data.
-
-runexocdf2nc.py
----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.exocdf2nc_parser
-   :prog: runexocdf2nc.py
+Starting from exported .csv file from KOR software, use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
diff --git a/doc/index.rst b/doc/index.rst
@@ -47,23 +47,24 @@ We have plans to support:
    config
    atmos
    waves
+   Processing data with runouts (use this!) <runots>
    aqd
-   wvs
    aqdhr
-   vec
-   sig
+   wvs
    rsk
+   sig
+   vec
+   wet
+   eofe
    exo
+   hobo
+   indexvel
    iq
+   lisst
    mc
-   wet
-   hobo
-   wxt
    sg
-   eofe
-   lisst
    tcm
-   indexvel
+   wxt
    turnaround
    code
    contributing

diff --git a/doc/iq.rst b/doc/iq.rst
@@ -9,25 +9,4 @@ depends on two :doc:`configuration files </config>`.
 IQ-exported Matlab file to raw .cdf
 ===================================
 
-First, export .mat files from the instrument software.
-
-Convert from exported .mat to a raw netCDF file with .cdf extension using ``runiqmat2cdf.py``.
-
-runiqmat2cdf.py
-----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.iqmat2cdf_parser
-   :prog: runiqmat2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into an CF-compliant netCDF file with .nc extension using ``runiqcdf2nc.py``.
-
-runiqcdf2nc.py
----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.iqcdf2nc_parser
-   :prog: runiqcdf2nc.py
+First, export .mat files from the instrument software. Then, use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
diff --git a/doc/lisst.rst b/doc/lisst.rst
@@ -1,29 +1,4 @@
 Sequoia Scientific LISST
 ************************
 
-Data will generally be processed using a series of run scripts. The first script for each instrument type
-depends on two :doc:`configuration files </config>`.
-
-Exported ASCII data to raw .cdf
-===============================
-
-Convert from comma-delimited file to a raw netCDF file with .cdf extension using ``runlisstcsv2cdf.py``.
-
-runlisstcsv2cdf.py
-------------------
-
-.. argparse::
-   :ref: stglib.core.cmd.lisstcsv2cdf_parser
-   :prog: runlisstcsv2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into CF-compliant netCDF file with .nc extension using ``runlisstcdf2nc.py``.
-
-runlisstcdf2nc.py
------------------
-
-.. argparse::
-   :ref: stglib.core.cmd.lisstcdf2nc_parser
-   :prog: runlisstcdf2nc.py
+Starting from exported comma-delimited file, use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
diff --git a/doc/mc.rst b/doc/mc.rst
@@ -1,29 +1,4 @@
 Seabird SBE 37 MicroCAT
 ***********************
-
-Data will generally be processed using a series of run scripts. The first script for each instrument type
-depends on two :doc:`configuration files </config>`.
 
-Instrument data to raw .cdf
-===========================
-
-Convert from exported .asc file to a raw netCDF file with .cdf extension using ``runmcasc2cdf.py``.
-
-runmcasc2cdf.py
-----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.mcasc2cdf_parser
-   :prog: runmcasc2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into a CF-compliant netCDF file with .nc extension using ``runmccdf2nc.py``.
-
-runmccdf2nc.py
----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.mccdf2nc_parser
-   :prog: runmccdf2nc.py
+Starting from exported .asc file, use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
diff --git a/doc/rsk.rst b/doc/rsk.rst
@@ -9,43 +9,15 @@ Instrument data to raw .cdf
 
 First, export data to text (.zip) format (File -> Export -> Text (\*.zip)) from the Ruskin software.
 
-Then use ``runrskcsv2cdf.py`` to convert this exported data to a raw netCDF .cdf file.
-
-runrskcsv2cdf.py
-----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.rskcsv2cdf_parser
-   :prog: runrskcsv2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into an CF-compliant netCDF file with .nc extension using ``runrskcdf2nc.py``, optionally including :doc:`atmospheric correction </atmos>` of the pressure data.
-
-runrskcdf2nc.py
----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.rskcdf2nc_parser
-   :prog: runrskcdf2nc.py
-
+Then use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.
 
 Waves processing
 ================
 
 Option 1: internal waves code
 -----------------------------
 
-This option is recommended as it does not require MATLAB. Generate the waves statistics and incorporate them into an CF-compliant netCDF file with .nc extension using ``runrsknc2waves.py``.
-
-runrsknc2waves.py
-~~~~~~~~~~~~~~~~~
-
-.. argparse::
-  :ref: stglib.core.cmd.rsknc2waves_parser
-  :prog: runrsknc2waves.py
-
+This option is recommended as it does not require MATLAB. Generate the waves statistics and incorporate them into an CF-compliant netCDF file with .nc extension using :doc:`runots </runots>`.
 
 Option 2: DIWASP
 ----------------

diff --git a/doc/runots.rst b/doc/runots.rst
@@ -0,0 +1,28 @@
+Processing data with runots
+***************************
+
+Data are processed using the umbrella run script ``runots``. With this script the user specifies the instrument and step to be applied to the data.
+
+For most instruments, this is a two-step process:
+
+1. instrument data to raw CDF
+2. raw CDF to clean .nc
+
+For some instruments (usually those measuring waves) there is a third step:
+
+3. clean .nc to wave-statistics .nc
+
+The user will call ``runots`` from the command line, with arguments described below. The autogenerated documentation below can be a little challenging to interpret, so here's an example:
+
+``runots aqd hdr2cdf gatts_file.txt config_file.yaml``
+
+The above command specifies ``aqd`` (Aquadopp collecting in currents mode) and ``hdr2cdf`` (step 1 above, instrument data to raw CDF), using the specified :doc:`configuration files </config>`.
+
+To get to the final .nc file, the user would then run ``runots aqd cdf2nc raw_cdf_file_name.cdf``
+
+runots
+------
+
+.. argparse::
+   :ref: stglib.core.cmd.runots_parser
+   :prog: runots
diff --git a/doc/sg.rst b/doc/sg.rst
@@ -1,29 +1,4 @@
 Seabird SBE 26plus Seagauge
 ***************************
-
-Data will generally be processed using a series of run scripts. The first script for each instrument type
-depends on two :doc:`configuration files </config>`.
 
-Instrument data to raw .cdf
-===========================
-
-Convert from exported .tide file to a raw netCDF file with .cdf extension using ``runsgtid2cdf.py``.
-
-runsgtid2cdf.py
-----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.sgtid2cdf_parser
-   :prog: runsgtid2cdf.py
-
-Raw .cdf to CF-compliant .nc
-============================
-
-Convert the raw .cdf data into a CF-compliant netCDF file with .nc extension using ``runsgcdf2nc.py``.
-
-runsgcdf2nc.py
----------------
-
-.. argparse::
-   :ref: stglib.core.cmd.sgcdf2nc_parser
-   :prog: runsgcdf2nc.py
+Starting from exported .tid file, use :doc:`runots </runots>` to process using the two :doc:`configuration files </config>`.