Update RRFS regression test cases

[hu5970]

Description

1. Change RRFS case from rrfs_3denvar_glbens to rrfs_3denvar_rdasens.
   The new RRFS case has the same configuration as RRFS_A warm cycles.
2. delete case netcdf_fv3_regional
3. add RRFS EnKF case for conventional observations: rrfs_enkf_conv

This PR depends on GSI-fix PR: NOAA-EMC/GSI-fix#20
This pull request is addressing Fix #586

Please copy the data for the new cases:
Hera: "/scratch1/BMC/wrfruc/mhu/code/data"
Cactus:"/lfs/h2/emc/ptmp/Ming.Hu/data"

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Those two new cases were tested on Hera and WCOSS2 (Cactus)

Checklist

• [x ] My code follows the style guidelines of this project
• [x ] I have performed a self-review of my own code
• [x ] I have commented my code, particularly in hard-to-understand areas
• [x ] New and existing tests pass with my changes
• [x ] Any dependent changes have been merged and published

[TingLei-NOAA]

@hu5970 I finished my first round of reviewing (:)) . I added a couple of general comments for consideration.
Another point i think should be noted is that, this PR changes GSI control of the regression tests itself, we can't depend on regression tests as for the regular PR for GSI codes. So, any comments/plan to further verify this PR 's changes on the regression test process?

[hu5970]

@TingLei-NOAA I do not have good method on how to test new added regression test cases. Only thing I can think of is to check output from the case to make sure the case look good based on the past experience.

[ShunLiu-NOAA]

@hu5970 Could you please start your regression test to ensure there is no impact of these changes on global and HAFS?

[hu5970]

Got regression test done on Hera:

[Ming.Hu@hfe05 build]$ ctest -j6
Test project /scratch1/BMC/wrfruc/mhu/code/gsi/GSI/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  495.86 sec
2/6 Test #2: rtma .............................   Passed  969.90 sec
3/6 Test #5: hafs_3denvar_hybens ..............   Passed  1104.37 sec
4/6 Test #6: global_enkf ......................   Passed  1164.92 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1282.50 sec
6/6 Test #1: global_4denvar ...................   Passed  1967.29 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1967.32 sec

The regression test is done on Dogwood:

[ming.hu@dlogin08 build] ctest -j6
Test project /lfs/h2/emc/ptmp/Ming.Hu/gsi/GSI/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  2349.54 sec
2/6 Test #6: global_enkf ......................   Passed  2417.15 sec
3/6 Test #2: rtma .............................   Passed  2657.60 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  2665.98 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  2835.76 sec
6/6 Test #1: global_4denvar ...................   Passed  3544.61 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 3544.62 sec

Test on Hercules:

[minghu@hercules-login-2 build]$ ctest -j6
Test project /work/noaa/wrfruc/mhu/code/gsi/GSI/build
    Start 3: rrfs_3denvar_rdasens
    Start 1: global_4denvar
    Start 2: rtma
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  489.81 sec
2/6 Test #6: global_enkf ......................   Passed  738.45 sec
3/6 Test #2: rtma .............................   Passed  967.07 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1095.57 sec
5/6 Test #4: hafs_4denvar_glbens ..............***Failed  1101.70 sec
6/6 Test #1: global_4denvar ...................   Passed  1682.71 sec

83% tests passed, 1 tests failed out of 6

Total Test time (real) = 1682.77 sec

The following tests FAILED:
	  4 - hafs_4denvar_glbens (Failed)
Errors while running CTest

Repeat failed case and it passed:

[minghu@hercules-login-2 build]$ ctest -R hafs_4denvar_glbens
Test project /work/noaa/wrfruc/mhu/code/gsi/GSI/build
    Start 4: hafs_4denvar_glbens
1/1 Test #4: hafs_4denvar_glbens ..............   Passed  1347.15 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 1347.16 sec

[hu5970]

To run EnKF for RRFS case, please change ENKF_MODE in ush/build.sh
from
ENKF_MODE=${ENKF_MODE:-"GFS"}
to
ENKF_MODE=${ENKF_MODE:-"FV3REG"}
and recompile both control and update GSI code.

The regression test with RRFS regional EnKF on Hera is:

[Ming.Hu@hfe05 build]$ ctest -j6
Test project /scratch1/BMC/wrfruc/mhu/code/gsi/GSI/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: rrfs_enkf_conv
1/6 Test #6: rrfs_enkf_conv ...................   Passed  485.47 sec
2/6 Test #3: rrfs_3denvar_rdasens .............   Passed  742.74 sec
3/6 Test #2: rtma .............................   Passed  1274.67 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1427.07 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1593.48 sec
6/6 Test #1: global_4denvar ...................   Passed  2463.73 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 2463.88 sec

Test on Hercules:

[minghu@hercules-login-2 build]$ ctest -j6
Test project /work/noaa/wrfruc/mhu/code/gsi/GSI/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: rrfs_enkf_conv
1/6 Test #6: rrfs_enkf_conv ...................   Passed  241.75 sec
2/6 Test #3: rrfs_3denvar_rdasens .............   Passed  489.68 sec
3/6 Test #2: rtma .............................   Passed  966.60 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1094.57 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1101.62 sec
6/6 Test #1: global_4denvar ...................   Passed  1862.56 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1862.57 sec