From 8f2a0bbf84cfed69d8854fda5c01ae062c104974 Mon Sep 17 00:00:00 2001 From: Matthias Mohr Date: Thu, 21 Dec 2023 01:00:39 +0100 Subject: [PATCH] Another draft --- optical.md | 513 ++++++++++++++++++++++++++++++++++------------------- 1 file changed, 333 insertions(+), 180 deletions(-) diff --git a/optical.md b/optical.md index 387b2ee..813373c 100644 --- a/optical.md +++ b/optical.md @@ -4,17 +4,17 @@ - **Identifier:** - **Field Name Prefix:** - - **Scope:** Item -- **Extension [Maturity Classification](https://github.com/radiantearth/stac-spec/tree/master/extensions/README.md#extension-maturity):** Proposal +- **Extension [Maturity Classification]:** Proposal - **Owner**: @m-mohr -This extension specifies how to create [STAC](https://github.com/radiantearth/stac-spec) Items that -comply to the [CEOS-ARD](http://ceos.org/ard/) product family specification for optical sensors: +This extension specifies how to create [STAC] Items that +comply to the [CEOS-ARD product family specifications] (PFS) for optical sensors: -- [Aquatic Reflectance (AR), 1.0](https://ceos.org/ard/files/PFS/AR/v1.0/CARD4L_Product_Family_Specification_Aquatic_Reflectance-v1.0.pdf) +- [Aquatic Reflectance (AR), 1.0] - LiDAR Terrain and Canopy Top Height, draft -- [Nighttime Lights Surface Radiance (NTSR), 1.0](https://ceos.org/ard/files/PFS/NLSR/v1.0/CARD4L_Product_Family_Specification_Nighttime_Light_Radiance-v1.0.pdf) -- [Surface Reflectance (SR), v5.0](https://ceos.org/ard/files/PFS/SR/v5.0/CARD4L_Product_Family_Specification_Surface_Reflectance-v5.0.pdf) -- [Surface Temperature (ST), v5.0](https://ceos.org/ard/files/PFS/ST/v5.0/CARD4L_Product_Family_Specification_Surface_Temperature-v5.0.pdf) +- [Nighttime Lights Surface Radiance (NTSR), 1.0] +- [Surface Reflectance (SR), v5.0] +- [Surface Temperature (ST), v5.0] **Additional resources:** @@ -33,24 +33,13 @@ comply to the [CEOS-ARD](http://ceos.org/ard/) product family specification for - [STAC Items](#stac-items) - [CEOS-ARD](#ceos-ard) - [Common Metadata](#common-metadata) - - [Fields](#fields) - - [Links](#links) - [Accuracy](#accuracy) - - [Fields](#fields-1) - - [Links](#links-1) - [EO (Electro-Optical)](#eo-electro-optical) - - [Fields](#fields-2) - - [Bands](#bands) - - [Links](#links-2) - [Processing](#processing) - - [Fields](#fields-3) - - [Links](#links-3) - [Projection](#projection) - [View](#view) - - [STAC Item Links](#stac-item-links) - - [STAC Item Assets](#stac-item-assets) - - [Additional Asset Properties](#additional-asset-properties) - - [raster:bands](#rasterbands) + - [Links](#links) + - [Assets](#assets) - [Notes](#notes) ## Document Structure @@ -66,25 +55,26 @@ The column *Field Name* refers to the STAC field names. The column *Req.* refers This profile doesn't define new STAC fields, it's just a profile that uses [existing STAC extensions](#stac-extensions) to map and fulfill the CEOS-ARD requirements. -| Name | Schema URI for `stac_extensions` | Required | -| ---- | -------------------------------- | -------- | -| [Accuracy](https://github.com/stac-extensions/accuracy) | `https://stac-extensions.github.io/accuracy/v1.0.0-beta.1/schema.json` | ✗ | -| [CEOS-ARD](https://github.com/stac-extensions/ceos-ard) | `https://stac-extensions.github.io/ceos-ard/v0.2.0/optical/schema.json` | ✓ | -| [Electro Optical](https://github.com/stac-extensions/eo) | `https://stac-extensions.github.io/eo/v1.1.0/schema.json` | ✓ | -| [File](https://github.com/stac-extensions/file) | `https://stac-extensions.github.io/file/v2.0.0/schema.json` | ✗ | -| [Processing](https://github.com/stac-extensions/processing) | `https://stac-extensions.github.io/processing/v1.1.0/schema.json` | ✗ | -| [Projection](https://github.com/stac-extensions/projection) | `https://stac-extensions.github.io/projection/v1.0.0/schema.json` | ✓ | -| [Raster](https://github.com/stac-extensions/raster) | `https://stac-extensions.github.io/raster/v1.1.0/schema.json` | ✓ | -| [View Geometry](https://github.com/stac-extensions/view) | `https://stac-extensions.github.io/view/v1.0.0/schema.json` | ✓ | +| Name | Schema URI for `stac_extensions` | Required | +| ----------------- | -------------------------------- | -------- | +| [Accuracy] | `https://stac-extensions.github.io/accuracy/v1.0.0-beta.1/schema.json` | ✗ | +| [CEOS-ARD] | `https://stac-extensions.github.io/ceos-ard/v0.2.0/optical/schema.json` | ✓ | +| [Classification] | `https://stac-extensions.github.io/classification/v1.1.0/schema.json` | ✗ | +| [Electro Optical] | `https://stac-extensions.github.io/eo/v2.0.0/schema.json` | ✓ | +| [File] | `https://stac-extensions.github.io/file/v2.0.0/schema.json` | ✗ | +| [Processing] | `https://stac-extensions.github.io/processing/v1.1.0/schema.json` | ✗ | +| [Projection] | `https://stac-extensions.github.io/projection/v1.0.0/schema.json` | ✓ | +| [Raster] | `https://stac-extensions.github.io/raster/v2.0.0/schema.json` | ✓ | +| [View Geometry] | `https://stac-extensions.github.io/view/v1.1.0/schema.json` | ✓ | ## STAC Collections CEOS-ARD lists a lot of requirements (and fields) that have common values across all generated STAC Items and assets. -Thus, it is **recommended** to provide a STAC Collection for the Items and put common fields (in the STAC Item `properties`) -into [Collection `summaries`](https://github.com/radiantearth/stac-spec/tree/v1.0.0/collection-spec/collection-spec.md#collection-fields). -While the STAC Item fields still need to be in the Item, too, you can de-duplicate links and assets by putting common -links once into the STAC Collection links. Also, common assets can be just put once into the STAC Collection using the -STAC extension [Collection Assets](https://github.com/radiantearth/stac-spec/tree/v1.0.0/collection-spec/collection-spec.md#assets). +Thus, it is **recommended** to provide a STAC Collection for the Items and +put common fields (in the STAC Item `properties`) into [Collection Summaries]. +While the STAC Item fields still need to be in the Item, too, +you can de-duplicate links and assets by putting common links once into the STAC Collection links. +Also, common assets can be just put once into the STAC Collection using [Collection Assets]. All this is still CEOS-ARD compliant as it doesn't require all information to be in a single file. ## STAC Items @@ -109,8 +99,6 @@ See the separate [CEOS-ARD extension README](README.md#fields) for details. ### Common Metadata -#### Fields - | Field Name | Req. | Description | | -------------- | ----- | ----------- | | license | *n/a* | Recommended to be specified in a STAC Collection. | @@ -121,70 +109,43 @@ See the separate [CEOS-ARD extension README](README.md#fields) for details. | constellation | 1.9 | Constellation name in lower-case. | | platform | 1.9 | Platform (mission) name in lower-case. | -#### Links - -| Relation Type | Req. | Description | -| ---------------------- | ---- | ----------- | -| platform | 1.9 | URL to a [CEOS Missions Database](https://database.eohandbook.com/database/missiontable.aspx) record. | -| instrument | 1.9 | URL to a [CEOS Instruments Database](https://database.eohandbook.com/database/instrumenttable.aspx) record. | -| measurement | 1.9 | URL to a [CEOS Measurements Database](https://database.eohandbook.com/measurements/overview.aspx) record. | -| instrument-calibration | 1.11 | URL to the instrument / sensor calibration parameters. | +Requirement 1.9 asks to provide links to: +- [CEOS Missions Database] record(s) +- [CEOS Instruments Database] record(s) +- [CEOS Measurements Database] record(s) ### Accuracy -#### Fields +The following fields may be provided: -| Field Name | Data Type | Req. | Description | -| --------------------------- | --------- | ---- | ----------- | -| accuracy:geometric_y_bias | number | 1.8 | An estimate of the northern geometric accuracy: Bias, in meters. | -| accuracy:geometric_y_stddev | number | 1.8 | An estimate of the northern geometric accuracy: Standard deviation, in meters. | -| accuracy:geometric_x_bias | number | 1.8 | An estimate of the eastern geometric accuracy: Bias, in meters. | -| accuracy:geometric_x_stddev | number | 1.8 | An estimate of the eastern geometric accuracy: Standard deviation, in meters. | -| accuracy:geometric_rmse | number | 1.8 | Radial root mean square error (rRMSE) for sub-sample accuracy, in meters. | +| Field Name | Req. | Description | +| --------------------------- | ---- | ----------- | +| accuracy:geometric_y_bias | 1.8 | An estimate of the northern geometric accuracy: Bias, in meters. | +| accuracy:geometric_y_stddev | 1.8 | An estimate of the northern geometric accuracy: Standard deviation, in meters. | +| accuracy:geometric_x_bias | 1.8 | An estimate of the eastern geometric accuracy: Bias, in meters. | +| accuracy:geometric_x_stddev | 1.8 | An estimate of the eastern geometric accuracy: Standard deviation, in meters. | +| accuracy:geometric_rmse | 1.8 | Radial root mean square error (rRMSE) for sub-sample accuracy, in meters. | -#### Links +The following links may be provided: -| Relation Type | Req. | Description | -| -------------------- | ---- | ----------- | -| geometric-correction | 1.7 | URL to the Geometric Correction algorithm details. | -| geometric-accuracy | 1.8 | URL describing the assessed geometric accuracy of the data. | -| radiometric-accuracy | 1.12 | URL describing the assessed absolute radiometric uncertainty of the data. | -| elevation-model | 1.14 | URLs to the Digital Elevation Model (DEM). | -| surface-model | 1.14 | URL to the Digital Surface Model (DSM). | +| Relation Type | Req. | Description | +| --------------- | ---- | ----------- | +| elevation-model | 1.14 | URLs to the Digital Elevation Model (DEM). | +| surface-model | 1.14 | URLs to the Digital Surface Model (DSM). | The relation types `elevation-model` and `surface-model` can be provided in any file format (e.g., HTML or PDF), but preferrably point to a STAC Collection or Item with additional metadata for the DEM/DSM. ### EO (Electro-Optical) -#### Fields - | Field Name | Req. | Description | | -------------- | ---- | ----------- | | eo:cloud_cover | 1.17 | **REQUIRED** in AR. Cloud cover as quality flag. | | eo:snow_cover | 1.17 | Snow/Ice cover as quality flag. | -| eo:bands | 1.10 | **REQUIRED.** All spectral bands must be specified in the assets. | More data quality flags than `eo:cloud_cover` and `eo:snow_cover` should usually be set for CEOS-ARD compliance, but other quality flag are not standardized in STAC yet. Please fall back to custom fields for now. -##### Bands - -**Deprecation Notice:** In a future version of the version of STAC and the EO extension `eo:bands` will be replaced by `bands`. -It is recommended to provide both for now. - -For `eo:bands` (soon: `bands`) it is **required** provide the `name` and `central_wavelength` (soon: `eo:central_wavelength`). -For AR it is also **required** to provide `full_width_half_max` (soon: `eo:full_width_half_max`). -Additional band information such as spectral response details are optional. - -#### Links - -| Relation Type | Req. | Description | -| ------------- | -------------------------- | ----------- | -| cloud | 1.17 / 2.5 | URL to documentation about the cloud detection. | -| cloud-shadow | 1.17 / 2.6 | URL to documentation about the cloud shadow detection. | -| snow-ice | 1.17 / 2.7 (ST) / 2.8 (SR) | URL to documentation about the snow and ice mask. | - ### Processing Requirement 1.13 **requires** all algorithms, and the sequence in which they were applied, to be identified in the metadata. @@ -196,7 +157,7 @@ This can be expressed in various ways: - A link to processing instructions that allow to execute the algorithms (relation type: `processing-expression`). This would cover requirements 1.13 and 1.15 at the same time. -#### Fields +The following fields may be provided: | Field Name | Req. | Description | | --------------------- | ----------- | ----------- | @@ -204,13 +165,13 @@ This can be expressed in various ways: | processing:expression | 1.13 / 1.15 | Machine-readable algorithms and/or processing chain. | | processing:lineage | 1.13 / 1.15 | Human-readable descriptions of the algorithms and/or processing chain. | -#### Links +The following links may be provided: -| Relation Type | Req. | Description | -| ---------------------- | ---- | ----------- | +| Relation Type | Req. | Description | +| ---------------------- | ----------- | ----------- | | processing-expression | 1.13 / 1.15 | A processing chain (or script) that describes how the data has been processed. | -| processing-description | 1.13 | A processing chain (or script) that describes how the data has been processed. | -| derived_from | 1.15 | Points back to the source's STAC Item. May be multiple items, if the product is derived from multiple acquisitions. | +| processing-description | 1.13 | A document that describes the algorithms, or an Algorithm Theoretical Basis document. | +| derived_from | 1.15 | Points back to the source's STAC Item. May be multiple items, if the product is derived from multiple acquisitions. | ### Projection @@ -226,102 +187,269 @@ It is recommended to provide both for now. ### View -| Field Name | Req. | Description | -| -------------------- | ----- | ------------------------------------------------------------ | -| view:off_nadir | *n/a* | The average off-nadir angle, for per-pixel angles. In degrees. | -| view:incidence_angle | \[2] | **REQUIRED.** The average incidence angle, for per-pixel angles, refer to the asset with the key `incidence-angle`. In degrees. | -| view:azimuth | \[2] | **REQUIRED.** The average azimuth angle, for per-pixel angles, refer to the asset with the key `azimuth`. In degrees. | -| view:sun_azimuth | \[2] | **REQUIRED.** The average sun azimuth angle, for per-pixel angles, refer to the asset with the key `sun-azimuth`. In degrees. | -| view:sun_elevation | \[2] | **REQUIRED.** The average sub elevation angle, for per-pixel angles, refer to the asset with the key `sun-elevation`. In degrees. | - -\[2] Requirement 2.8 for Surface Temperature (ST) / 2.11 for Surface Reflectance (SR) - -### STAC Item Links - -In addition to the links defined in specific extensions above, the links can or must be specified: - -| Relation Type | Req. | Description | -| ------------------------- | ------------------- | ------------------------------------------------------------ | -| related | 1.14 | URL to the sources of auxiliary data used in the generation process. This is **required** if auxiliary data is used in the generation process. Excludes DEMs and DSMs, which must use [separate relation types](#accuracy) instead. | -| cloud | 1.17 / 2.5 | URL to documentation about the cloud detection. | -| cloud-shadow | 1.17 / 2.6 | URL to documentation about the cloud shadow detection. | -| snow-ice | 1.17 / 2.7 (ST) / 2.8 (SR) | URL to documentation about the snow and ice mask. | -| land-water | 2.7 (SR) | URL to documentation about the land and water mask (SR only). | -| atmosphere-emissivity | 3.2 (ST) | **REQUIRED.** URL to documentation about corrections for atmosphere and emissivity (ST only). | -| measurement-normalization | 3.3 (SR) | URL to documentation about measurement normalization (SR only). | -| atmospheric-scattering | 3.4 (SR) | **REQUIRED.** URL to documentation about the directional atmospheric scattering algorithms (SR only). | -| water-vapor | 3.5 (SR) | **REQUIRED.** URL to documentation about the water vapour corrections (SR only). | -| ozone | 3.6 (SR) | URL to documentation about the ozone corrections (SR only). | - -### STAC Item Assets +| Field Name | Req. | Description | +| -------------------- | --------- | ----------- | +| view:incidence_angle | see below | **REQUIRED for AR, ST and SR.** The average incidence angle. | +| view:azimuth | see below | **REQUIRED for AR, ST and SR.** The average azimuth angle. | +| view:sun_azimuth | see below | **REQUIRED for AR, ST and SR.** The average sun azimuth angle. | +| view:sun_elevation | see below | **REQUIRED.** The average sun elevation angle. | +| view:moon_azimuth | see below | **REQUIRED for NLSR.** The average moon azimuth angle. [**TBD**](https://github.com/stac-extensions/view/pull/7) | +| view:moon_elevation | see below | **REQUIRED for NLSR.** The average moon elevation angle. [**TBD**](https://github.com/stac-extensions/view/pull/7) | + +The requirements regarding solar, lunar and viewing geometry have different requirement numbers: +- AR: 2.12 +- NLSR: 2.11 and 2.16 +- SR: 2.11 +- ST: 2.8 + +If provided, all values **must** be in degrees. + +### Custom Fields + +The following fields are not defined in a STAC extension yet, +but we'd like to propose them to the STAC community: + +| Field Name | Data Type | Req. | Description | +| ----------------- | --------- | ----------- | ----------- | +| moon:illumination | number | 2.14 (NLSR) | **REQUIRED for NLSR.** The average moon illumination, in %. | + +### Links + +In addition to the links defined in specific extensions above, the following relation types may be used: + +| Relation Type | Req. | Description | +| ------------- | --------- | ----------- | +| related | 1.14 | URL to the sources of auxiliary data used in the generation process, ideally as STAC Items. | +| describedby | *various* | URL to documentation, see below for details. | + +#### related + +Links to the sources of auxiliary data used in the generation process. +This is **required** if auxiliary data is used in the generation process. +Excludes DEMs and DSMs, which must use [separate relation types](#accuracy) instead. + +#### describedby + +Various CEOS-ARD requirements ask for documentation about some specifics of the data. +Those links should use the relation type `describedby` and have a clear title that clearly gives information about what the link documents. +Although we recommend to use `describedby` as relation type, +implementors can choose to use another relation type such as `about` if it suits their implementation better. +The requested documentation can in principle all be available through a single link, +so you don't necessarily need to provide one link per bullet point. + +The following links are **required**: +- 3.2 (ST): Documentation about corrections for atmosphere and emissivity. +- 3.4 (AR): Documentation about the atmospheric reflectance correction. +- 3.4 (SR): Documentation about the directional atmospheric scattering algorithms. +- 3.4 (NLSR): Documentation about the atmospheric corrections. +- 3.5 (AR/SR): Documentation about the water vapour corrections. +- 3.5 (NLSR): Documentation about the lunar radiance corrections. +- 3.6 (AR): Documentation about the ozone corrections. +- 3.6 (NLSR): Documentation about the stray light corrections. +- 3.7 (AR): Documentation about other trace faseous absorption corrections. + +The following links are **optional**: +- 1.7 (all): Geometric Correction algorithm details. +- 1.8 (all): Description of the assessed geometric accuracy of the data. +- 1.11 (all): Instrument / sensor calibration parameters. +- 1.12 (all): Description of the assessed absolute radiometric uncertainty of the data. +- 1.17 (all) / 2.5 (all): Documentation about the cloud detection. +- 1.17 (all) / 2.6 (all): Documentation about the cloud shadow detection. +- 1.17 (all) / 2.7 (ST) / 2.8 (AR/NLSR/SR): Documentation about the (snow and) ice mask. +- 2.7 (AR/NLSR/SR): Documentation about the land and water mask. +- 3.2 (AR/SR/NLSR): Information about the measurement uncertainty. +- 3.3 (AR/SR/NLSR): Documentation about measurement normalization. +- 3.6 (SR): Documentation about the ozone corrections. +- 3.11 (AR): Information on adjacency effect correction. +- 3.12 (AR): Information on floating vegetation/surface scum water mask and/or correction +- 3.13 (AR): Information on turbid water mask and/or correction + +### Assets **Data Access:** Requirement 1.16 **requires** information about data access. This requirement is automatically fulfilled by STAC if the referenced assets are publicly accessible. If authentication or other steps are required to access the data, this should at least be described in the Collection or Item description. -For a more machine-readable way implementors could also use other means, such as the -[Authentication extension](https://github.com/stac-extensions/authentication). - -**Per-Pixel Metadata:** Whether the metadata is provided in a single record relevant to all pixels, -or separately for each pixel, is at the discretion of the data provider. +For a more machine-readable way implementors could also use other means, such as the [Authentication] extension. **Roles:**: The role names specify the values to be used in the Asset's `roles`. Each of the assets can either be exposed individually or grouped together in any form. In the latter case the role names can simply be merged to a set of unique role names. Roles can also be combined for a single file. -For example, a cloud mask which is also including cloud shadows -can use the roles `cloud` and `cloud-shadow` for a single file. -The `values` in `raster:bands` property can the specify which value(s) correspond to clouds and -which value(s) correspond to cloud shadows respectively. -The *italic* role names could be used as the asset's key. - -**Additional Properties:** The **bold** additional properties are required. - -| Role Name(s) | Req. | Additional properties | Description | -| -------------------------------------- | -------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | -| data | 3.1 / 2.2 | `type`, `created`, **`eo:bands`**, **`raster:bands`** (with `data_type`, `bits_per_sample` and **`nodata`**), `file:byte_order` | **REQUIRED.** Points to the actual measurements. The value(s) for pixels that do not correspond to an observation must be provided in the property `nodata` in `raster:bands`. | -| *date*, metadata | 1.3 | `type` | Points to a file with per-pixel acquisition timestamps. | -| *incomplete-testing*, metadata | 2.3 | `type`, **`raster:bands`** (with **`values`**) | **REQUIRED.** Points to a file that identifies pixels for which the per-pixel tests have not all been successfully completed. See CEOS-ARD req. 2.3 for details. | -| *saturation*, metadata | 2.4 | `type`, `eo:bands`, **`raster:bands`** (with **`values`**) | **REQUIRED.** Points to a file that indicates where pixels in the input spectral bands are saturated. If the saturation is given per band, either `eo:bands` or `values` in `raster:bands` is **required** to indicate which pixels are saturated for each spectral band. | -| *cloud*, metadata | 2.5 | `type`, **`raster:bands`** (with **`values`**) | **REQUIRED.** Points to a file that indicates whether a pixel is assessed as being cloud. | -| *cloud-shadow*, metadata | 2.6 | `type`, **`raster:bands`** (with **`values`**) | **REQUIRED.** Points to a file that indicates whether a pixel is assessed as being cloud shadow. | -| *snow-ice*, metadata | 2.7 (ST) / 2.8 (SR) | `type`, **`raster:bands`** (with **`values`**) | Points to a file that indicates whether a pixel is assessed as being snow/ice or not. | -| *land-water*, metadata | 2.7 (SR) | `type`, **`raster:bands`** (with **`values`**) | Points to a file that indicates whether a pixel is assessed as being land or water. | -| *incidence-angle*, metadata | 2.8 (ST) / 2.11 (SR) | `type`, **`raster:bands`** (with `data_type`, `bits_per_sample` and **`unit`**), `file:byte_order` | Points to a file with per-pixel incidence angles. `unit` is usually `degree`. | -| *azimuth*, metadata | 2.8 (ST) / 2.11 (SR) | `type`, **`raster:bands`** (with `data_type`, `bits_per_sample` and **`unit`**), `file:byte_order` | Points to a file with per-pixel azimuth angles. `unit` is usually `degree`. | -| *sun-azimuth*, metadata | 2.8 (ST) / 2.11 (SR) | `type`, **`raster:bands`** (with `data_type`, `bits_per_sample` and **`unit`**), `file:byte_order` | Points to a file with per-pixel sun azimuth angles. `unit` is usually `degree`. | -| *sun-elevation*, metadata | 2.8 (ST) / 2.11 (SR) | `type`, **`raster:bands`** (with `data_type`, `bits_per_sample` and **`unit`**), `file:byte_order` | Points to a file with per-pixel sun elevation angles. `unit` is usually `degree`. | -| *terrain-shadow*, metadata | 2.9 (SR) | `type`, **`raster:bands`** (with **`values`**) | Points to a file that indicates whether a pixel is not directly illuminated due to terrain shadowing. | -| *terrain-occlusion*, metadata | 2.10 (SR) | `type`, **`raster:bands`** (with **`values`**) | Points to a file that indicates whether a pixel is not visible to the sensor due to terrain occlusion during off-nadir viewing. | -| *terrain-illumination*, metadata | 2.12 (SR) | `type`, `raster:bands` (with `data_type`, `bits_per_sample`), `file:byte_order` | Points to a file with coefficients used for terrain illumination correction are provided for each pixel. | - -Note: `raster:bands[*].values` is not standardized yet in STAC, this could change to -`file:values` or something different with a similar structure in the future. +For example, a cloud mask which also includes cloud shadows can use the roles `cloud` and `cloud-shadow` for a single file. +The `classification:classes` and/or `bands` properties can specify which value(s) correspond to clouds and/or cloud shadows respectively. + +**Media Types:** It is STRONGLY RECOMMENDED to provide the [Media Type] of the file format for each asset. + +#### Data + +For non-metadata requirements that refer to how the data must be created and processed +please consult the relevant CEOS-ARD specification. + +There can be one or multiple data assets, e.g. one file per band or all bands in one file. + +For data assets, the following properties are relavant in the context of CEOS-ARD: + +| Field Name | Req. | Description | +| ---------- | ----- | ----------- | +| roles | *n/a* | **REQUIRED.** All data assets must to include the `data` role. | +| bands | 1.10 | **REQUIRED.** All (spectral) bands must be specified in the assets as [Band Object]s. | + +##### Bands + +**Deprecation Notice:** In this document we only refer to the new `bands` construct that will be introcued with STAC 1.1. +For backward compatibility, it is recommended to also provide `eo:bands` and `raster:bands`. +Please check the [Raster] and [Electro Optical] extensions for details. + +| Field Name | Req. | DDescription | +| ---------- | ---- | ------------ | +| nodata | 2.2 | **REQUIRED.** Value(s) for no-data. | +| unit | 3.1 | The unit of the values in the asset, preferably compliant to [UDUNITS-2] units (singular). For ST the unit must be `kelvin`, for NLSR the unit is usually `nit`. | + +It is **required** provide the `name` and `eo:central_wavelength`. +For AR it is also **required** to provide `eo:full_width_half_max`. +Additional band information such as spectral response details are optional. + +See the STAC [Bands] for further details. + +#### Per-Pixel Metadata + +This section describes required and optional per-pixel metadata. +Per-pixel metadata can be encoded in different way, for example: +1. multiple files with a mask per file (only requires the corresponding roles to be set, see below) +2. a single file with a mask per band (requires `bands`, see [Bands] in [Common metadata]) +3. a single-band file with masks as bit-fields (requires `classification:bitfields`, see [Classification] extension) + +In any way, the per-pixel metadata and its values must be clearly identifiable, e.g. by providing +clear titles, roles, band information, bitfields, classes, and/or unit where applicable. + +Some general rules that apply for per-pixel metadata assets: +- **Roles:** All metadata assets are **required** to include the role `metadata` in the `roles` array. +- **Types:** It is STRONGLY RECOMMENDED to provide the media type in the `type` property for all assets. + +##### Incomplete Testing / Per-Pixel Assessment + +- **PFS:** all +- **Requirement:** 2.3 +- **Role:** `incomplete-testing` + +A per-pixel mask **must** be provided that identifies the pixel for which the per-pixel metadata check have not completed. +The asset **must** identify the values of the mask using the `classification:classes` or `classification:bitfield` properties. +The mask **may** be provided per band, in this case the `bands` property must be provided. + +##### Saturation + +- **PFS:** all +- **Requirement:** 2.4 +- **Role:** `saturation` + +A per-pixel mask **must** be provided that identifies pixel for which at least one band is saturated. +The asset **must** identify the values of the mask using the `classification:classes` or `classification:bitfield` properties. +The mask **may** be provided per band, in this case the `bands` property must be provided. + +##### Cloud / Cloud Shadow + +- **PFS:** all +- **Requirement:** 2.5 / 2.6 +- **Role:** `cloud` / `cloud-shadow` + +Separate per-pixel masks **must** be provided that identify pixels which are identified as cloud or cloud shadow respecitvely. +The asset **must** identify the values of the mask using the `classification:classes` or `classification:bitfield` properties. + +The following assets are **required** for some of the PFS: + +| Role Name(s) | Req. (PFS) | Description | +| -------------------- | -------------------- | ----------- | +| date | 1.3 | Per-pixel acquisition timestamps. | +| snow-ice | 2.7 (ST) / 2.8 (SR) | Indicates whether a pixel is assessed as being snow/ice or not. | +| land-water | 2.7 (SR) | Indicates whether a pixel is assessed as being land or water. | +| incidence-angle | 2.8 (ST) / 2.11 (SR) | Per-pixel incidence angles. `unit` is usually `degree`. | +| azimuth | 2.8 (ST) / 2.11 (SR) | Per-pixel azimuth angles. `unit` is usually `degree`. | +| sun-azimuth | 2.8 (ST) / 2.11 (SR) | Per-pixel sun azimuth angles. `unit` is usually `degree`. | +| sun-elevation | 2.8 (ST) / 2.11 (SR) | Per-pixel sun elevation angles. `unit` is usually `degree`. | +| terrain-shadow | 2.9 (SR) | Indicates whether a pixel is not directly illuminated due to terrain shadowing. | +| terrain-occlusion | 2.10 (SR) | Indicates whether a pixel is not visible to the sensor due to terrain occlusion during off-nadir viewing. | +| terrain-illumination | 2.12 (SR) | Coefficients used for terrain illumination correction are provided for each pixel. | + +- Land/Water Mask: 2.7 (AR/NLSR) v +- Snow/Ice Mask 2.8 (NLSR) v +- Sea/Lake/River Ice Mask: 2.8 (AR) +- Sun Glint: 2.9 (AR) +- Whitecap Foam Mask: 2.11 (AR) +- Floating Vegetation / Surface Scum Mask / Correction: 2.14 / 3.12 (AR) +- Aerosol Optical Depth Parameter: 2.15 (AR) +- Optically Deep or Optically Shallow Assessment: 2.17 (AR) +- Turbid Water Flag / Corection: 2.18 / 3.13 (AR) +- Bidirectional Reflectance Distribution Function Applied: 2.19 (AR) +- Altitude (ASL): 2.20 (AR) +- Brightness Temperature: 2.15 (NLSR) +- Solar Zenith Angle: 2.16 (NLSR) + +##### Optional + +- Land/Water Mask: 2.7 (SR) ^ +- Snow/Ice Mask 2.8 (SR) / 2.7 (ST) ^ +- Terrain Shadow Mask: 2.9 (SR/NLSR) +- Terrain Occlusion: 2.10 (SR/NLSR) +- Solar / Lunar / View Geometry: 2.11 (SR/NLSR) / 2.8 (ST) / 2.12 (AR) +- Terrain Illumination Correction: 2.12 (SR/NLSR) +- Sky Glin: 2.10 (AR) +- Adjacency Effects: 2.13 (AR) +- Deep / Shallow Water: 2.16 (AR) +- Measurement Uncertainty: 3.3 (ST) / 3.2 (SR/NLSR) +- Sun Glint Correction: 3.8 (AR) +- Sky Glint Correction: 3.9 (AR) + +The following assets are **optional** for some or all of the PFS: + +| Role Name(s) | Req. (PFS) | Description | +| -------------------- | -------------------- | ----------- | +| date | 1.3 | Per-pixel acquisition timestamps. | +| snow-ice | 2.7 (ST) / 2.8 (SR) | Indicates whether a pixel is assessed as being snow/ice or not. | +| land-water | 2.7 (SR) | Indicates whether a pixel is assessed as being land or water. | +| incidence-angle | 2.8 (ST) / 2.11 (SR) | Per-pixel incidence angles. `unit` is usually `degree`. | +| azimuth | 2.8 (ST) / 2.11 (SR) | Per-pixel azimuth angles. `unit` is usually `degree`. | +| sun-azimuth | 2.8 (ST) / 2.11 (SR) | Per-pixel sun azimuth angles. `unit` is usually `degree`. | +| sun-elevation | 2.8 (ST) / 2.11 (SR) | Per-pixel sun elevation angles. `unit` is usually `degree`. | +| terrain-shadow | 2.9 (SR) | Indicates whether a pixel is not directly illuminated due to terrain shadowing. | +| terrain-occlusion | 2.10 (SR) | Indicates whether a pixel is not visible to the sensor due to terrain occlusion during off-nadir viewing. | +| terrain-illumination | 2.12 (SR) | Coefficients used for terrain illumination correction are provided for each pixel. | #### Additional Asset Properties -The following table lists properties that may occur in the assets. -The list doesn't specify which fields apply to which asset and it also doesn't specify which fields are required. -For those details please refer to the ["Additional properties" column in the table above](#stac-item-assets). +For per-pixel metadata assets, the following properties may be relavant in the context of CEOS-ARD: + +| Field Name | Req. | Description | +| ---------- | ----- | ----------- | +| roles | *n/a* | **REQUIRED.** All data assets must to include the `data` role. | +| bands | 1.10 | **REQUIRED.** All bands must be specified in the assets as [Band Object]s. | + +##### Bands + +**Deprecation Notice:** In this document we only refer to the new `bands` construct that will be introcued with STAC 1.1. +For backward compatibility, it is recommended to also provide `eo:bands` and `raster:bands`. +Please check the [Raster] and [Electro Optical] extensions for details. -| Field Name | Req. | Data Type | Description | -| --------------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | -| type | *n/a* | string | STRONGLY RECOMMENDED. The [media type](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md#asset-media-type) of the file format. | -| created | *n/a* | string | The time of the processing is specified via the `created` property of the asset as specified in the [STAC Common metadata](https://github.com/radiantearth/stac-spec/tree/v1.0.0/item-spec/common-metadata.md#date-and-time). | -| eo:bands | 1.10 | \[[Band Object](https://github.com/stac-extensions/eo/blob/v1.0.0/README.md#band-object)\] | see [Bands in EO](#eo-electro-optical) | -| raster:bands | see below | \[[Raster Band Object](https://github.com/stac-extensions/raster/blob/v1.1.0/README.md#raster-band-object)\] | Bands with at least the required fields for the corresponding asset role (see above and below). | -| file:byte_order | *n/a* | string | One of `big-endian` or `little-endian`. | +| Field Name | Req. | DDescription | +| ---------- | ---- | ------------ | +| nodata | 2.2 | Value(s) for no-data. | +| unit | 3.1 | The unit of the values in the asset, preferably compliant to [UDUNITS-2] units (singular). | +| classification:classes | *n/a* | Lists the value that are in the file and describes their meaning. | -##### raster:bands +See the STAC [Bands] for further details. -| Field Name | Req. | Data Type | Description | -| --------------- | ----- | ------------------------------------------------------------ | ------------------------------------------------------------ | -| data_type | *n/a* | string | One of the [Data Types](https://github.com/stac-extensions/raster/blob/v1.1.0/README.md#data-types). | -| unit | *n/a* | string | The unit of the values in the asset, preferably compliant to [UDUNITS-2](https://ncics.org/portfolio/other-resources/udunits2/) units (singular). | -| bits_per_sample | *n/a* | integer | Actual number of bits per sample (e.g., 8, 16, 32, ...) | -| values | *n/a* | \[[Mapping Object](https://github.com/stac-extensions/file/blob/v2.1.0/README.md#mapping-object)\] | Lists the value that are in the file and describes their meaning. | -| nodata | 2.2 | \[any] | Value(s) for no-data. | +The following table lists properties that may occur in the assets. +The list doesn't specify which fields apply to which asset and it also doesn't specify which fields are required. +For those details please refer to the ["Additional properties" column in the table above](#assets). + +##### bands -*ToDo: values should be replaced by the classification extension.* +| Field Name | Req. | Description | +| ---------------------- | ----- | ------------- | +| data_type | *n/a* | One of the [Data Types]. | +| unit | *n/a* | The unit of the values in the asset, preferably compliant to [UDUNITS-2] units (singular). | +| bits_per_sample | *n/a* | Actual number of bits per sample (e.g., 8, 16, 32, ...) | +| classification:classes | *n/a* | Lists the value that are in the file and describes their meaning. | +| nodata | 2.2 | Value(s) for no-data. | ## Notes @@ -329,16 +457,41 @@ Some additional notes on the requirements - Requirements 1.1, 1.2, and 2.1 are generally covered by implementing and publishing STAC metadata for the data. - 1.12 (AR): CEOS-ARD requires the number of bits. - This is not reflected in the STAC profile and must be added by implementors manually to be fully compliant (unless CEOS-ARD gets updated). -- 2.13 (SR): CEOS-ARD lists no specific requirements thus it's missing in this document. -- 3.2 (SR) / 3.3 (ST): Measurement Uncertainty is not required and it was not clear in which form this should be provided. - Also the CEOS-ARD specification states for SR that - - > "\[i]n current practice, users determine fitness for purpose based on knowledge of the lineage of the data, - > rather than on a specific estimate of measurement uncertainty." - - Thus this requirement is not captured in this document. -- 3.3 (SR): Measurement Normalisation is not required and it was not clear in which form this should be provided. - Thus this requirement is not captured in this document, but we have added the link relation type - `measurement-normalization` to link to information on measurement normalisation. -- 4.1 (SR): CEOS-ARD lists no specific requirements thus it's missing in this document. + This is not reflected in the STAC profile and must be added by implementors manually + via the `bits_per_sample` property to be fully compliant + (unless CEOS-ARD gets updated as it looks like an error to us). +- 2.13 (SR/NLSR): CEOS-ARD lists no specific metadata requirements. +- Requirement 4.1 lists no specific metadata requirements, but refers back to 1.x requirements. + +[CEOS-ARD product family specifications]: +[Aquatic Reflectance (AR), 1.0]: +[Nighttime Lights Surface Radiance (NTSR), 1.0]: +[Surface Reflectance (SR), v5.0]: +[Surface Temperature (ST), v5.0]: + +[CEOS Missions Database]: +[CEOS Instruments Database]: +[CEOS Measurements Database]: + +[STAC]: +[Maturity Classification]: +[Common metadata]: +[Media Type]: +[Collection Summaries]: +[Collection Assets]: +[Bands]: +[Band Object]: +[Data Types]: + +[Accuracy]: +[Authentication]: +[Classification]: +[CEOS-ARD]: +[Electro Optical]: +[File]: +[Processing]: +[Projection]: +[Raster]: +[View Geometry]: + +[UDUNITS-2]: