Update to latest master

docs/userguide/src/site/index.md

@@ -28,7 +28,7 @@ Some of the technology in the TDS:
 * An integrated server provides data access through the [OpenGIS Consortium (OGC) Web Coverage Service (WCS)](https://www.ogc.org/standards/wcs){:target="_blank"} protocol, for any gridded dataset whose coordinate system information is complete.
 * An integrated server provides data access through the [OpenGIS Consortium (OGC) Web Map Service (WMS)](http://www.opengeospatial.org/standards/wms){:target="_blank"} protocol, for any gridded dataset whose coordinate system information is complete.
   This software was developed by Jon Blower (University of Reading (UK) E-Science Center) as part of the [ESSC Web Map Service for environmental data](https://github.com/Reading-eScience-Centre/edal-java){:target="_blank"} (aka Godiva3).
-* The integrated [ncISO server](iso_metadata.html) provides automated metadata analysis and ISO metadata generation.
+* The optional [ncISO server](iso_metadata.html) provides automated metadata analysis and ISO metadata generation.
 * The integrated [NetCDF Subset Service](netcdf_subset_service_ref.html) allows subsetting certain CDM datasets in coordinate space, using a REST API.
   Gridded data subsets can be returned in [CF-compliant](http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html){:target="_blank"} netCDF-3 or netCDF-4.
   Point data subsets can be returned in CSV, XML, WaterML (with [assistance](https://github.com/Unidata/thredds/tree/5.0.0/waterml#waterml){:target="_blank"}

docs/userguide/src/site/pages/tds_tutorial/metadata/IsoMetadataRef.md

@@ -6,30 +6,31 @@ toc: false
 permalink: iso_metadata.html
 ---
 
-The TDS distribution includes the [ncISO](https://github.com/Unidata/threddsIso/){:target="_blank"} package from NOAA/Environmental Data Management (many thanks to Dave Neufeld and Ted Habermann).
+The TDS can include ISO metadata services by using the [ncISO plugin](https://github.com/Unidata/threddsIso/){:target="_blank"} package from NOAA/Environmental Data Management (many thanks to Dave Neufeld and Ted Habermann).
+See [enabling ncISO services](adding_ogc_iso_services.html) for more information on how to add these services to your TDS.
 
-## `ncISO` Services
-`ncISO` supports three new services for datasets:
+## ncISO Services
+The ncISO plugin provides three new services for datasets:
 * `NCML`: an NcML representation of the dataset's structure and metadata;
 * `ISO`: an ISO 19115 metadata representation of the dataset; and
 * `UDDC`: an evaluation of how well the metadata contained in the dataset conforms to the NetCDF Attribute Convention for Data Discovery (NACDD) (see the [NOAA/EDM page on NACDD](http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery){:target="_blank"}).
 
-## Enabling `ncISO` Services
+## Enabling ncISO Services
 
-The `ncISO` services are enabled by default. 
-These services can be disabled for locally served datasets by including the following in the `threddsConfig.xml` file:
+The ncISO services are disabled by default.
+Provided that you have added the [ncISO plugin](adding_ogc_iso_services.html#nciso-configuration), these services can be enabled for locally served datasets by including the following in the `threddsConfig.xml` file:
 
 ~~~xml
 <NCISO>
-  <ncmlAllow>false</ncmlAllow>
-  <uddcAllow>false</uddcAllow>
-  <isoAllow>false</isoAllow>
+  <ncmlAllow>true</ncmlAllow>
+  <uddcAllow>true</uddcAllow>
+  <isoAllow>true</isoAllow>
 </NCISO>
 ~~~
 
-## Providing `ncISO` Services For Datasets
+## Providing ncISO Services For Datasets
 
-Once `ncISO` is enabled, datasets can be configured to have the three `ncISO` services in the TDS catalog configuration files similar to the way other services are configured.
+Once ncISO is enabled, datasets can be configured to have the three ncISO services in the TDS catalog configuration files similar to the way other services are configured.
 The `service` element's `serviceType` and `base` attribute values must be as follows: 
 
 ~~~xml

docs/userguide/src/site/pages/tds_tutorial/tds_configuration/AddingOgcIsoServices.md

@@ -6,11 +6,12 @@ toc: false
 permalink: adding_ogc_iso_services.html
 ---
 
-## Configure TDS To Allow WCS, WMS, and `ncISO` Access
+## Configure TDS To Allow WCS, WMS, and ncISO Access
 
-Out of the box, the TDS distribution will have `WCS`, `WMS`, and `ncISO` enabled.
+Out of the box, the TDS distribution will have `WCS` and `WMS` enabled.
 If you do not wish to use these services, they must be explicitly allowed in the `threddsConfig.xml` file.
-Please see the  [threddsConfig.xml file](tds_config_ref.html#wcs-service) documentation for information on how to disable these services.
+The ncISO services are disabled by default but can be enabled by adding the plugin and updating the `threddsConfig.xml` file.
+Please see the [threddsConfig.xml file](tds_config_ref.html#wcs-service) documentation for information on how to enable/disable these services.
 The default `threddsConfig.xml` file (which should now be in your `${tds.content.root.path}/content/thredds` directory) contains commented out sections for each of these services.
 
 ### `WCS` Configuration
@@ -41,19 +42,29 @@ The following section in the `threddsConfig.xml` file controls the WMS service:
 Additional `WMS` configuration options can be set in the `threddsConfig.xml` file,
 More details are available in the `WMS` section of the [threddsConfig.xml file](tds_config_ref.html#wms-service) documentation.
 
-### `ncISO` Configuration
+### ncISO Configuration
 
-The following section in the `threddsConfig.xml` file controls the `ncIso` services:
+#### Adding the plugin
+To use the ncISO services, you must add the `tds-plugin-jar-with-dependencies.jar` artifact to your TDS for TDS versions >= 5.5.
+For TDS versions prior to 5.5 this artifact was included in the TDS war file.
+To see which versions of the plugin are compatible with your TDS version see the table [here](https://github.com/Unidata/threddsIso).
+The plugin can be downloaded on the [TDS downloads page](https://downloads.unidata.ucar.edu/tds/){:target="_blank"}.
+The downloaded ncISO plugin jar file should be placed in your `${tomcat_home}/webapps/thredds/WEB-INF/lib/` directory.
+
+#### Updating `threddsConfig.xml`
+The following section in the `threddsConfig.xml` file controls the ncISO services:
 
 ~~~xml
 <NCISO>
-  <ncmlAllow>true</ncmlAllow>
-  <uddcAllow>true</uddcAllow>
-  <isoAllow>true</isoAllow>
+  <ncmlAllow>false</ncmlAllow>
+  <uddcAllow>false</uddcAllow>
+  <isoAllow>false</isoAllow>
 </NCISO>
 ~~~
 
-Each `*Allow` element allows one of the three `ncISO` services.
+Each `*Allow` element allows one of the three ncISO services.
+
+After adding the ncISO plugin and updating your `threddsConfig.xml`, the TDS should be restarted.
 
 ### Adding `WCS` And `WMS` Services
 
@@ -70,9 +81,9 @@ Adding them to an existing compound service would look something like this:
 </service>
 ~~~
 
-### Adding `ncISO` Services
+### Adding ncISO Services
 
-Similar to above, as long as the `ncISO` services are enabled, all that is required for the TDS to provide `ncISO` services on datasets is for those datasets to reference the `ncISO` service elements.
+Similar to above, as long as the ncISO services are enabled, all that is required for the TDS to provide ncISO services on datasets is for those datasets to reference the ncISO service elements.
 For instance, adding to the same compound service as above:
 
 ~~~xml

docs/userguide/src/site/pages/thredds/TdsServices.md

@@ -21,10 +21,7 @@ The TDS has a set of **Standard Data Services** that are always available (unles
 * opendap
 * wcs
 * wms
-* iso
-* ncml
-* uddc
-
+
 ### Available Services
 
 The TDS configures the appropriate set of standard data services for each dataType/featureType.
@@ -71,20 +68,20 @@ Consult the [Configuration Catalogs](config_catalog.html) documentation and [TDS
 | Remote THREDDS Catalog Service | *Disabled* | `/thredds/remoteCatalogService` |
 
 
-## `ncISO` Metadata Services
+## ncISO Metadata Services
+
+The ncISO plugin adds the *ncISO Metadata Services*, which facilitate the generation of [ISO 19115 metadata representation](https://en.wikipedia.org/wiki/Geospatial_metadata){:target="_blank"} from data in [NetCDF](https://www.unidata.ucar.edu/software/netcdf/).
 
-*`ncISO` Metadata Services* facilitate the generation of [ISO 19115 metadata representation](https://en.wikipedia.org/wiki/Geospatial_metadata){:target="_blank"} from data in [NetCDF](https://www.unidata.ucar.edu/software/netcdf/). 
+  {% include note.html content="
+  Learn how to [enable the ncISO services](adding_ogc_iso_services.html#nciso-configuration).
+  " %}
 
-The three `ncISO` Metadata Services are:
+The three ncISO Metadata Services are:
 
 * **ISO**
 
   The ISO Metadata Service provides [ISO 19115 metadata representation](https://www.ngdc.noaa.gov/wiki/index.php/NcISO){:target="_blank"} of a dataset's structure and metadata.
-
-   {% include note.html content="
-  Learn how to [enable NcISO](adding_ogc_iso_services.html) in the TDS Configuration file.
-  " %}
-
+
 * **NCML**
 
     The *NCML Metadata Service* provides NCML representation of a dataset.
@@ -103,9 +100,9 @@ The three `ncISO` Metadata Services are:
 |-----------------|-------------------|-------------------------------|
 | Service Name         | Default Availability | Access Point   |
 |:----------------|:------------------|:------------------------------|
-| ISO | Enabled | `/thredds/iso/*`  |
-| NCML | Enabled | `/thredds/ncml/*`  |
-| UDDC | Enabled | `/thredds/uddc/*`  |
+| ISO | Disabled | `/thredds/iso/*`  |
+| NCML | Disabled | `/thredds/ncml/*`  |
+| UDDC | Disabled | `/thredds/uddc/*`  |
 
 #### Catalog Service Configuration:
 (These are the exact [required](services_ref.html#tds-requirements-for-thredds-catalog-service-elements) values to enable these service.)

docs/userguide/src/site/pages/thredds/ThreddsConfigRef.md

@@ -299,15 +299,16 @@ Here is the description of the various options:
   Optional; default is that there is no size limitation.
   If the file is > 2 GB, large format netCDF will be written.
 
-### ncISO Service
+### ncISO Services
 
-By default, these services are enabled, and can be disabled by including the following in the `threddsConfig.xml` file:
+By default, these services are disabled.
+Provided that you have added the [ncISO plugin](adding_ogc_iso_services.html#nciso-configuration), these services can be enabled by including the following in the `threddsConfig.xml` file:
 
 ~~~xml
 <NCISO>
-  <ncmlAllow>false</ncmlAllow>
-  <uddcAllow>false</uddcAllow>
-  <isoAllow>false</isoAllow>
+  <ncmlAllow>true</ncmlAllow>
+  <uddcAllow>true</uddcAllow>
+  <isoAllow>true</isoAllow>
 </NCISO>
 ~~~
 

gradle/root/publishing.gradle

@@ -113,6 +113,25 @@ def publishDownloadsTask = tasks.register('publishDownloads', PublishToRawRepoTa
 
 publish.dependsOn publishDownloadsTask
 
+def publishReleaseInfoAndLatestTask = tasks.register('publishReleaseInfoAndLatest', PublishToRawRepoTask) {
+    group = 'publishing'
+    description = 'Publish release_info.json and startup/latest.xml to Nexus downloads.'
+    host = 'https://artifacts.unidata.ucar.edu/'
+    repoName = 'downloads-tds'
+
+    publishSrc = new File(rootProject.getProjectDir(), "project-files/downloads/")
+
+    onlyIf {
+        // Will be evaluated at task execution time, not during configuration.
+        // Fails the build if the specified properties haven't been provided.
+        username = getPropertyOrFailBuild NEXUS_USERNAME_KEY
+        password = getPropertyOrFailBuild NEXUS_PASSWORD_KEY
+        return true
+    }
+}
+
+publish.dependsOn publishReleaseInfoAndLatestTask
+
 // The "publish" tasks require credentials for our Nexus server, which they look for in Gradle properties.
 // If those properties (i.e. NEXUS_USERNAME_KEY and NEXUS_PASSWORD_KEY) haven't been provided, the build will fail.
 // Therefore, we only want to configure credentials when a "publish" task is part of the execution plan. Otherwise,

project-files/downloads/release_info.json

@@ -0,0 +1,78 @@
+{
+  "package": "tds",
+  "packageName": "THREDDS Data Server",
+  "releases": [
+    {
+      "version": "5.5",
+      "status": "beta",
+      "description": "This is the current version running at thredds.ucar.edu. It contains several bug fixes compared to version 5.4.",
+      "baseURL": "https://artifacts.unidata.ucar.edu/repository/downloads-tds/5.5/",
+      "resources": {
+        "tds_war": {
+          "type" : "web application",
+          "package": "thredds-5.5-SNAPSHOT.war",
+          "name": "TDS (Web Application Archive)",
+          "hash": {
+            "MD5": "thredds-5.5-SNAPSHOT.war.md5",
+            "SHA1": "thredds-5.5-SNAPSHOT.war.sha1",
+            "SHA256": "thredds-5.5-SNAPSHOT.war.sha256"
+          },
+          "info": "THREDDS Data Server (TDS).<br><strong>IMPORTANT:</strong> Before deploying, rename to thredds##5.5-SNAPSHOT.war or thredds.war"
+        },
+        "tdm_jar": {
+          "type": "library",
+          "package": "tdm-5.5-SNAPSHOT.jar",
+          "name": "TDM (Jar file)",
+          "hash": {
+            "MD5": "tdm-5.5-SNAPSHOT.jar.md5",
+            "SHA1": "tdm-5.5-SNAPSHOT.jar.sha1",
+            "SHA256": "tdm-5.5-SNAPSHOT.jar.sha256"
+          },
+          "info": "THREDDS Data Manager (TDM) for the latest snapshot release (version 5.5 SNAPSHOT) "
+        },
+        "thredds_iso_jar": {
+          "type": "library",
+          "package": "tds-plugin-2.4.6-SNAPSHOT-jar-with-dependencies.jar",
+          "name": "THREDDS ISO Plugin (Jar file)",
+          "hash": {
+            "MD5": "tds-plugin-2.4.6-SNAPSHOT-jar-with-dependencies.jar.md5",
+            "SHA1": "tds-plugin-2.4.6-SNAPSHOT-jar-with-dependencies.jar.sha1",
+            "SHA256": "tds-plugin-2.4.6-SNAPSHOT-jar-with-dependencies.jar.sha256"
+          },
+          "info": "THREDDS ISO Plugin for the latest snapshot release (version 2.4.6 SNAPSHOT) "
+        }
+      }
+    },
+    {
+      "version": "5.4",
+      "releaseDate": "2022-07-01",
+      "status": "current",
+      "description": "",
+      "baseURL": "https://artifacts.unidata.ucar.edu/repository/downloads-tds/5.4/",
+      "resources": {
+        "tds_war": {
+          "type" : "web application",
+          "package": "thredds-5.4.war",
+          "name": "TDS (Web Application Archive)",
+          "hash": {
+            "MD5": "thredds-5.4.war.md5",
+            "SHA1": "thredds-5.4.war.sha1",
+            "SHA256": "thredds-54.war.sha256"
+          },
+          "info": "THREDDS Data Server (TDS).<br><strong>IMPORTANT:</strong> Before deploying, rename to thredds##5.4.war or thredds.war"
+        },
+        "tdm_jar": {
+          "type": "library",
+          "package": "tdm-5.4.jar",
+          "name": "TDM (Jar file)",
+          "hash": {
+            "MD5": "tdm-5.4.md5",
+            "SHA1": "tdm-5.4.jar.sha1",
+            "SHA256": "tdm-5.4.jar.sha256"
+          },
+          "info": "THREDDS Data Manager (TDM) for the current release (version 5.4) "
+        }
+      }
+    }
+  ]
+}

project-files/downloads/startup/latest.xml

@@ -0,0 +1,4 @@
+<tdsVersions>
+    <version name="development" value="5.5-SNAPSHOT"/>
+    <version name="release" value="5.4"/>
+</tdsVersions>

tds-platform/build.gradle

@@ -81,14 +81,15 @@ dependencies {
     runtime 'org.apache.logging.log4j:log4j-web'
 
     // tds
-    api 'org.json:json:20230227'
+    api 'org.json:json:20231013'
     api 'com.coverity.security:coverity-escapers:1.1.1'
     api 'org.thymeleaf:thymeleaf-spring5:3.1.2.RELEASE'
     api 'jakarta.validation:jakarta.validation-api'
     api "org.hibernate.validator:hibernate-validator:${depVersion.hibernateValidator}"
     api "org.hibernate.validator:hibernate-validator-annotation-processor:${depVersion.hibernateValidator}"
     api 'jakarta.el:jakarta.el-api'
     runtime 'org.glassfish:javax.el:3.0.0'
+    runtime "jaxen:jaxen:${depVersion.jaxen}"
 
     // waterml stuff
     api 'org.apache.xmlbeans:xmlbeans'
@@ -106,11 +107,6 @@ dependencies {
     // gwt version defined in gradle/any/shared-mvn-coords.gradle, accessible via gradle/any/dependencies.gradle
     api "com.google.gwt:gwt-dev:${depVersion.gwt}"
     api "com.google.gwt:gwt-user:${depVersion.gwt}"
-
-    // threddsIso
-    runtime 'EDS:tds-plugin:2.4.6-SNAPSHOT'
-    runtime 'net.sf.saxon:Saxon-HE:9.7.0-8'
-    runtime "jaxen:jaxen:${depVersion.jaxen}"
   }
 }
 

tds-testing-platform/build.gradle

@@ -28,6 +28,10 @@ dependencies {
     api 'EDS:tds-plugin:2.4.0-SNAPSHOT' // api because of :it
     api 'net.sf.saxon:Saxon-HE:9.7.0-8' // api because of :it
 
+    // dap4
+    // defined in tds-platform as well, but using runtime config
+    api 'org.apache.logging.log4j:log4j-web' // api because of :dap4:d4tests)
+
     // tds
     api "jaxen:jaxen:${depVersion.jaxen}"  // api because of XPath operations in mock tests
 

tds/build.gradle

@@ -65,6 +65,8 @@ dependencies {
 
   // Needed for XPath operations in mock tests
   testCompile 'jaxen:jaxen'
+  // Needed for XPaths in WmsDetailedConfig
+  runtime 'jaxen:jaxen'
 
   // edal ncwms related libs
   compile('uk.ac.rdg.resc:edal-common')
@@ -73,11 +75,6 @@ dependencies {
   compile ('uk.ac.rdg.resc:edal-wms')
   compile('uk.ac.rdg.resc:edal-godiva')
 
-  // threddsIso related libs
-  runtime 'EDS:tds-plugin'
-  runtime 'net.sf.saxon:Saxon-HE'
-  runtime 'jaxen:jaxen'
-
   // json writing
   'org.json:json'
 

tds/src/integrationTests/java/thredds/server/catalog/TestServiceDefaults.java

@@ -39,7 +39,7 @@ public void testStandardServices() throws IOException {
     Service s = ds.getServiceDefault();
     Assert.assertNotNull(s);
     Assert.assertTrue(s.getType() == ServiceType.Compound);
-    Assert.assertEquals(10, s.getNestedServices().size());
+    Assert.assertEquals(7, s.getNestedServices().size());
   }
 
   // Relies on:
@@ -62,7 +62,7 @@ public void testStandardServicesDatasetScan() throws IOException {
     Assert.assertNotNull(s);
 
     Assert.assertTrue(s.getType() == ServiceType.Compound);
-    Assert.assertEquals(10, s.getNestedServices().size());
+    Assert.assertEquals(7, s.getNestedServices().size());
   }
 
   @Test
@@ -72,7 +72,7 @@ public void testUserDefinedServices() throws IOException {
     Assert.assertEquals(3, cat.getServices().size());
 
     check(cat, "all", 11);
-    check(cat, "GridServices", 10);
+    check(cat, "GridServices", 7);
     check(cat, "opendapOnly", 1);
 
     Service localServices = cat.findService("opendapOnly");