EO Exploitation Platform Common Architecture
+<Template Service> Design Document
+EOEPCA.SDD.xxx
COMMENTS and ISSUES |
+PDF |
+
EUROPEAN SPACE AGENCY CONTRACT REPORT |
+TELESPAZIO VEGA UK Ltd |
+
+
-
+
- AMENDMENT HISTORY +
-
+
This document shall be amended by releasing a new edition of the document in its entirety.
+
+The Amendment Record Sheet below records the history and issue status of this document.+
+Table 1. Amendment Record Sheet ++ + ++ + + + + + +ISSUE +DATE +REASON ++ + +0.1
dd/mm/yyyy
Initial in-progress draft
+
+ + +
1. Introduction
+1.1. Purpose and Scope
+This document presents the <Template Service> Design for the Common Architecture.
+1.2. Structure of the Document
+-
+
- Section 2 - Overview +
-
+
Provides an over of the <Template Service> component, within the context of the wider Common Architecture design.
+
+ - Section 3 - [mainDesign] +
-
+
Provides the design of the <Template Service> component.
+
+
1.3. Reference Documents
+The following is a list of Reference Documents with a direct bearing on the content of this document.
+| Reference | +Document Details | +Version | +
|---|---|---|
| + |
+ EOEPCA - Use Case Analysis |
+
+ Issue 1.0, |
+
| + |
+ Exploitation Platform - Functional Model, |
+
+ Issue 1.0, |
+
| + |
+ Thematic Exploitation Platform Open Architecture, |
+
+ Issue 1, |
+
| + |
+ OGC Testbed-14: WPS-T Engineering Report, |
+
+ 18-036r1, |
+
| + |
+ OGC WPS 2.0 REST/JSON Binding Extension, Draft, |
+
+ 1.0-draft + |
+
| + |
+ Common Workflow Language Specifications, |
+
+ v1.0.2 + |
+
| + |
+ OGC Testbed-13, EP Application Package Engineering Report, |
+
+ 17-023, |
+
| + |
+ OGC Testbed-13, Application Deployment and Execution Service Engineering Report, |
+
+ 17-024, |
+
| + |
+ OGC Testbed-14, Application Package Engineering Report, |
+
+ 18-049r1, |
+
| + |
+ OGC Testbed-14, ADES & EMS Results and Best Practices Engineering Report, |
+
+ 18-050r1, |
+
| + |
+ OpenSearch GEO: OpenSearch Geo and Time Extensions, |
+
+ 10-032r8, |
+
| + |
+ OpenSearch EO: OGC OpenSearch Extension for Earth Observation, |
+
+ 13-026r9, |
+
| + |
+ OGC EO Dataset Metadata GeoJSON(-LD) Encoding Standard, |
+
+ 17-003r1/17-084 + |
+
| + |
+ OGC OpenSearch-EO GeoJSON(-LD) Response Encoding Standard, |
+
+ 17-047 + |
+
| + |
+ The Payment Card Industry Data Security Standard, |
+
+ v3.2.1 + |
+
| + |
+ CEOS OpenSearch Best Practise, |
+
+ v1.2, |
+
| + |
+ OpenID Connect Core 1.0, |
+
+ v1.0, |
+
| + |
+ OGC Catalogue Services 3.0 Specification - HTTP Protocol Binding (Catalogue Services for the Web), |
+
+ v3.0, |
+
| + |
+ OGC Web Map Server Implementation Specification, |
+
+ v1.3.0, |
+
| + |
+ OGC Web Map Tile Service Implementation Standard, |
+
+ v1.0.0, |
+
| + |
+ OGC Web Feature Service 2.0 Interface Standard – With Corrigendum, |
+
+ v2.0.2, |
+
| + |
+ OGC Web Coverage Service (WCS) 2.1 Interface Standard - Core, |
+
+ v2.1, |
+
| + |
+ Web Coverage Processing Service (WCPS) Language Interface Standard, |
+
+ v1.0.0, |
+
| + |
+ Amazon Simple Storage Service REST API, |
+
+ API Version 2006-03-01 + |
+
1.4. Terminology
+The following terms are used in the Master System Design.
+| Term | +Meaning | +
|---|---|
Admin |
+User with administrative capability on the EP |
+
Algorithm |
+A self-contained set of operations to be performed, typically to achieve a desired data manipulation. The algorithm must be implemented (codified) for deployment and execution on the platform. |
+
Analysis Result |
+The Products produced as output of an Interactive Application analysis session. |
+
Analytics |
+A set of activities aimed to discover, interpret and communicate meaningful patters within the data. Analytics considered here are performed manually (or in a semi-automatic way) on-line with the aid of Interactive Applications. |
+
Application Artefact |
+The 'software' component that provides the execution unit of the Application Package. |
+
Application Deployment and Execution Service (ADES) |
+WPS-T (REST/JSON) service that incorporates the Docker execution engine, and is responsible for the execution of the processing service (as a WPS request) within the ‘target’ Exploitation Platform. |
+
Application Descriptor |
+A file that provides the metadata part of the Application Package. Provides all the metadata required to accommodate the processor within the WPS service and make it available for execution. |
+
Application Package |
+A platform independent and self-contained representation of a software item, providing executable, metadata and dependencies such that it can be deployed to and executed within an Exploitation Platform. Comprises the Application Descriptor and the Application Artefact. |
+
Bulk Processing |
+Execution of a Processing Service on large amounts of data specified by AOI and TOI. |
+
Code |
+The codification of an algorithm performed with a given programming language - compiled to Software or directly executed (interpretted) within the platform. |
+
Compute Platform |
+The Platform on which execution occurs (this may differ from the Host or Home platform where federated processing is happening) |
+
Consumer |
+User accessing existing services/products within the EP. Consumers may be scientific/research or commercial, and may or may not be experts of the domain |
+
Data Access Library |
+An abstraction of the interface to the data layer of the resource tier. The library provides bindings for common languages (including python, Javascript) and presents a common object model to the code. |
+
Development |
+The act of building new products/services/applications to be exposed within the platform and made available for users to conduct exploitation activities. Development may be performed inside or outside of the platform. If performed outside, an integration activity will be required to accommodate the developed service so that it is exposed within the platform. |
+
Discovery |
+User finds products/services of interest to them based upon search criteria. |
+
Execution |
+The act to start a Processing Service or an Interactive Application. |
+
Execution Management Service (EMS) |
+The EMS is responsible for the orchestration of workflows, including the possibility of steps running on other (remote) platforms, and the on-demand deployment of processors to local/remote ADES as required. |
+
Expert |
+User developing and integrating added-value to the EP (Scientific Researcher or Service Developer) |
+
Exploitation Tier |
+The Exploitation Tier represents the end-users who exploit the services of the platform to perform analysis, or using high-level applications built-in on top of the platform’s services |
+
External Application |
+An application or script that is developed and executed outside of the Exploitation Platform, but is able to use the data/services of the EP via a programmatic interface (API). |
+
Guest |
+An unregistered User or an unauthenticated Consumer with limited access to the EP’s services |
+
Home Platform |
+The Platform on which a User is based or from which an action was initiated by a User |
+
Host Platform |
+The Platform through which a Resource has been published |
+
Identity Provider (IdP) |
+The source for validating user identity in a federated identity system, (user authentication as a service). |
+
Interactive Application |
+A stand-alone application provided within the exploitation platform for on-line hosted processing. Provides an interactive interface through which the user is able to conduct their analysis of the data, producing Analysis Results as output. Interactive Applications include at least the following types: console application, web application (rich browser interface), remote desktop to a hosted VM. |
+
Interactive Console Application |
+A simple Interactive Application for analysis in which a console interface to a platform-hosted terminal is provided to the user. The console interface can be provided through the user’s browser session or through a remote SSH connection. |
+
Interactive Remote Desktop |
+An Interactive Application for analysis provided as a remote desktop session to an OS-session (or directly to a 'native' application) on the exploitation platform. The user will have access to a number of applications within the hosted OS. The remote desktop session is provided through the user’s web browser. |
+
Interactive Web Application |
+An Interactive Application for analysis provided as a rich user interface through the user’s web browser. |
+
Key-Value Pair |
+A key-value pair (KVP) is an abstract data type that includes a group of key identifiers and a set of associated values. Key-value pairs are frequently used in lookup tables, hash tables and configuration files. |
+
Kubernetes (K8s) |
+Container orchestration system for automating application deployment, scaling and management. |
+
Login Service |
+An encapsulation of Authenticated Login provision within the Exploitation Platform context. The Login Service is an OpenID Connect Provider that is used purely for authentication. It acts as a Relying Party in flows with external IdPs to obtain access to the user’s identity. |
+
EO Network of Resources |
+The coordinated collection of European EO resources (platforms, data sources, etc.). |
+
Object Store |
+A computer data storage architecture that manages data as objects. Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier. |
+
On-demand Processing Service |
+A Processing Service whose execution is initiated directly by the user on an ad-hoc basis. |
+
Platform (EP) |
+An on-line collection of products, services and tools for exploitation of EO data |
+
Platform Tier |
+The Platform Tier represents the Exploitation Platform and the services it offers to end-users |
+
Processing |
+A set of pre-defined activities that interact to achieve a result. For the exploitation platform, comprises on-line processing to derive data products from input data, conducted by a hosted processing service execution. |
+
Processing Result |
+The Products produced as output of a Processing Service execution. |
+
Processing Service |
+A non-interactive data processing that has a well-defined set of input data types, input parameterisation, producing Processing Results with a well-defined output data type. |
+
Products |
+EO data (commercial and non-commercial) and Value-added products and made available through the EP. It is assumed that the Hosting Environment for the EP makes available an existing supply of EO Data |
+
Resource |
+A entity, such as a Product, Processing Service or Interactive Application, which is of interest to a user, is indexed in a catalogue and can be returned as a single meaningful search result |
+
Resource Tier |
+The Resource Tier represents the hosting infrastructure and provides the EO data, storage and compute upon which the exploitation platform is deployed |
+
Reusable Research Object |
+An encapsulation of some research/analysis that describes all aspects required to reproduce the analysis, including data used, processing performed etc. |
+
Scientific Researcher |
+Expert user with the objective to perform scientific research. Having minimal IT knowledge with no desire to acquire it, they want the effort for the translation of their algorithm into a service/product to be minimised by the platform. |
+
Service Developer |
+Expert user with the objective to provide a performing, stable and reliable service/product. Having deeper IT knowledge or a willingness to acquire it, they require deeper access to the platform IT functionalities for optimisation of their algorithm. |
+
Software |
+The compilation of code into a binary program to be executed within the platform on-line computing environment. |
+
Systematic Processing Service |
+A Processing Service whose execution is initiated automatically (on behalf of a user), either according to a schedule (routine) or triggered by an event (e.g. arrival of new data). |
+
Terms & Conditions (T&Cs) |
+The obligations that the user agrees to abide by in regard of usage of products/services of the platform. T&Cs are set by the provider of each product/service. |
+
Transactional Web Processing Service (WPS-T) |
+Transactional extension to WPS that allows adhoc deployment / undeployment of user-provided processors. |
+
User |
+An individual using the EP, of any type (Admin/Consumer/Expert/Guest) |
+
Value-added products |
+Products generated from processing services of the EP (or external processing) and made available through the EP. This includes products uploaded to the EP by users and published for collaborative consumption |
+
Visualisation |
+To obtain a visual representation of any data/products held within the platform - presented to the user within their web browser session. |
+
Web Coverage Service (WCS) |
+OGC standard that provides an open specification for sharing raster datasets on the web. |
+
Web Coverage Processing Service (WCPS) |
+OGC standard that defines a protocol-independent language for the extraction, processing, and analysis of multi-dimentional coverages representing sensor, image, or statistics data. |
+
Web Feature Service (WFS) |
+OGC standard that makes geographic feature data (vector geospatial datasets) available on the web. |
+
Web Map Service (WMS) |
+OGC standard that provides a simple HTTP interface for requesting geo-registered map images from one or more distributed geospatial databases. |
+
Web Map Tile Service (WMTS) |
+OGC standard that provides a simple HTTP interface for requesting map tiles of spatially referenced data using the images with predefined content, extent, and resolution. |
+
Web Processing Services (WPS) |
+OGC standard that defines how a client can request the execution of a process, and how the output from the process is handled. |
+
Workspace |
+A user-scoped 'container' in the EP, in which each user maintains their own links to resources (products and services) that have been collected by a user during their usage of the EP. The workspace acts as the hub for a user’s exploitation activities within the EP |
+
1.5. Glossary
+The following acronyms and abbreviations have been used in this report.
+| Term | +Definition | +
|---|---|
AAI |
+Authentication & Authorization Infrastructure |
+
ABAC |
+Attribute Based Access Control |
+
ADES |
+Application Deployment and Execution Service |
+
ALFA |
+Abbreviated Language For Authorization |
+
AOI |
+Area of Interest |
+
API |
+Application Programming Interface |
+
CMS |
+Content Management System |
+
CWL |
+Common Workflow Language |
+
DAL |
+Data Access Library |
+
EMS |
+Execution Management Service |
+
EO |
+Earth Observation |
+
EP |
+Exploitation Platform |
+
FUSE |
+Filesystem in Userspace |
+
GeoXACML |
+Geo-specific extension to the XACML Policy Language |
+
IAM |
+Identity and Access Management |
+
IdP |
+Identity Provider |
+
JSON |
+JavaScript Object Notation |
+
K8s |
+Kubernetes |
+
KVP |
+Key-value Pair |
+
M2M |
+Machine-to-machine |
+
OGC |
+Open Geospatial Consortium |
+
PDE |
+Processor Development Environment |
+
PDP |
+Policy Decision Point |
+
PEP |
+Policy Enforcement Point |
+
PIP |
+Policy Information Point |
+
RBAC |
+Role Based Access Control |
+
REST |
+Representational State Transfer |
+
SSH |
+Secure Shell |
+
TOI |
+Time of Interest |
+
UMA |
+User-Managed Access |
+
VNC |
+Virtual Network Computing |
+
WCS |
+Web Coverage Service |
+
WCPS |
+Web Coverage Processing Service |
+
WFS |
+Web Feature Service |
+
WMS |
+Web Map Service |
+
WMTS |
+Web Map Tile Service |
+
WPS |
+Web Processing Service |
+
WPS-T |
+Transactional Web Processing Service |
+
XACML |
+eXtensible Access Control Markup Language |
+
2. Overview
+TBD
+Unresolved directive in index.adoc - include::03.design/00.design.adoc[]
++
<< End of Document >>
+