High Level Design

oslc-service: An Express.js Middleware Component for OSLC services

oslc-service is an Express.js middleware component that can be used to provide OSLC services to any Node/Express Web application. oslc-service provides OSLC services by implementing the OSLC capabilities defined in the OASIS OSLC Core specification.

A typical Express Web application would include OSLC services using code similar to the following:

// The NODE_ENV environment variable is often not properly set, default to development	process.env.NODE_ENV = process.env.NODE_ENV || 'development';
var env = require('./config/env.js');

// Load, configure and initialize the Express module given its configuration
var express = require('./config/express');
var app = express();

app.use(function(err, req, res, next){
  console.error(err.stack);
  res.send(500, 'oslc-server could not handle the request');
});
app.listen(env.listenPort, env.listenHost);
module.exports = app;
console.log('oslc-server running on port:'+env.listenPort);

Environment and other configuration information is provided in env.js. This configuration information would usually include:

the environment in which the application is intended to run (dev, test, production),
The HTTP scheme, host and port to use for communication
the starting service provider catalog describing the OSLC domains and services the server should be configured with,
configuration information for the underlying database, and
any other configuration information that might be specific to the application. Configuration information for Express itself is in config/express.js. This contains the setup for other Express middleware components for authentication, logging, body parsing, entity body compression, session management, dynamic document processing, and static document access. The express.js config file includes OSLC middleware services just like any other Express middleware:

    module.exports = function() {
        var app = express();
        app.use(oslcService(env));
        return app;
    }

app.use(oslcService(env)); configures the OSLC services and sets the express routes for any LDPCs and discovery resources that are defined by the configuration information.

Key features

oslc-service supports the following features:

Supports OSLC v2 and v3
Supports containers that define root LDPCs for projects and any supported OSLC domains (oslc-cm, oslc-rm, oslc-qm, oslc-am)
OSLC Discovery (both LDPC Link header based discovery and using OSLC discovery resources)
Create, Read, Update and Delete OSLC resources for any domain using LDP services
Resource preview
Delegated dialogs for creation and selection of OSLC resources with default dialogs derived from domain vocabularies and resource shape constraints
Query Capability that enables OSLC queries on resources
Creation Factory that allows users to POST resources corresponding to a Resource Shape
Dynamically configured at startup based on a service provider catalog RDF configuration file
- supported domains with dynamic configuration based on chosen domain vocabularies and shapes
- default root containers (created if missing)
- root LDPCs (create any that are missing on startup)
Implemented as an Express.js middleware component to exploit dynamic and asynchronous capabilities of JavaScript and Node
Provides an architecture for implementing OSLC adapters to existing data sources:
- Utilizes an LDP API for LDP capabilities and persistence of resources as needed
- Implemented on a abstract storage capability where concrete implementations provide adapters for other data sources (similar to Lyo Designer ConnectorManager.java)

Future features:

Actions and Automation
Tracked Resource Sets

The rest of this Wiki contains notes on the implementation of, and documentation for the use of oslc-service.

Governance

oslc-service, like all the OSLC4JS projects is Open Source software using the Apache v2 license.

OSLC/oslc-service GitHub Project: provides source code management
Wiki This Wiki: provides project design and detailed documentation
Issues: used for project and issue management
npm package The Node module is published as an npm package
install with npm install oslc-service

High Level Design

oslc-services is an express middleware component that can be used to add OSLC capabilities on resources in an Express.js application. OSLC4JS Architecture

OSLC Core 3.0 leverages LDP to formalize the resources that an OSLC server provides. This reduces the variability of OSLC servers by having predictable resource formats (Turtle and JSON-LD) so that applications can understand the state of the resource representation, and how to find the links to other potentially interesting resources by leveraging RDF.

However, an OSLC server is not required to be a general purpose LDP server. Rather an OSLC server needs to decide how it organizes and implements its resources. OSLC servers are required to support LDPCs for resource containment, but they are not required to support any LDPC container (BasicContainer, DirectContainer, IndirectContainer), they are only required to support the container they need.

The implementation of oslc-service cannot be a simple delegation to ldp-service adapter middleware since the interaction between oslc-service and LDP as well as its required storage services does not fit a simple chain of command pattern. For example, creating a new resource and adding it to container involves multiple interactions depending on how the LDPC is implemented. So ldp-service uses an abstract storage.js services interface to specify its storage service requirements, and then different storage service implementations implement this abstract interface.

For this reason the oslc-service does not delegate to ldp-service, it uses ldp.js, an LDP API that is also used by ldp-service to build OSLC capabilities on LDP.

This will allow oslc-service to be implemented simply, implementing only those parts of LDP that are needed to meet the conformance criteria in the OSLC Core 3.0 specification, and not providing additional LDP capabilities that would complicate the oslc-service, or create containers it doesn't need or can't use.

There can be many oslc-service instantiations on different storage services used by the same oslc-server. Since oslc-service depends on storage services (for resource preview, attachments, OSLC 2.0 discovery resources, OSLC query, selective properties), it is the oslc-server that will instantiate the storage services for a particular route, providing that implementation to the oslc-service that uses it.

Persistence Strategy

oslc-service defines an abstract storage.js module that specifies its generic storage services needs. This abstract module can be instantiated for concrete implementations on specific data sources such as Apache Jena, supported by ldp-service-jena. Adaptors for other data sources would provide concrete implementations that instantiate this ldp-service/storage.js module.

Any user of oslc-service will need to specify the storage services concrete implementation in the env.storageService configuration parameter. This allows an application to establish different routes to different oslc-service instances using different data sources.

Configuration

Configuration consists of two basic parts: server environment configuration based on development phase, and express middleware configuration.

env.js is responsible exploring the environment to determine server configuration information. It gets the configuration from var config = require('./config.js'); which loads a JavaScript file containing configuration information for the desired environment (development, test, production) as specified in process.env.NODE_ENV.

For example, development.js specifies the configuration information that will be loaded by env.js when process.env.NODE_ENV=development:

module.exports = {
	// Development configuration settings
	"scheme": "http",
	"host": "localhost",
	"port": 3000,
	"context": "/",
	"storageImpl": "storage-jena",
	"jenaURL": "http://localhost:3030/univ/",

	"services": path.resolve("./services/spc.ttl"),
	"contentType": "text/turtle"
}

The config properties are:

scheme - http or https
host - the host name
port - the http port
context - the root URI pathname for LDP resources serviced by this configuration. Other context roots could use a different storageImpl.
storageImpl - the concrete implementation of the storage.js services to use (e.g. storage-jena)
jenaURL - for storage-jena, the URL of the Fuseki TDB2 dataset that provides the graph database to use. For example, ./fuseki-server --config=../config-univ.ttl starts the Fuseki server using the dataset in the config-univ.ttl file.

An HTTP server that uses Express middleware has to configure the middleware components and routes that provide the intended services. The app.js or express.js express configuration file would typically set the following routes:

Serving static HTML pages from ./public folder
Setting the views folder and view engine for processing dynamic HTML pages
Loading and initializing the storage services
Setting up routes to oslc-servics middleware components based on the environment configuration
route to handle errors - writes an error to the console, and does res.send(500, 'Something broke!');

In addition, oslc-service has some specific routes it adds:

route to fill in req.fullURL = env.appBase + req.originalUrl. appBase is the scheme//host:port from the env/config information.
Route to fill in req.rawBody which is used in the LDP service PUT and POST methods to read the entity request body.
sets app.route(env.context + '*') , env.context is the ldpBase URL path name. Meaning the req.originalUrl is what matched this pathname, not the fullURL
provides all the route.http methods (route.get(function(req, res, next), etc.) these all end in req.end(), and never call the route next() function.

Authentication

Authentication would typically be supported by Express middleware components such as passport.js. Any server that wants to integrate with the IBM jazz.net applications would need to provide a jazz.net rootservices document, and minimally support OAuth1.0a.

OSLC and LDP CRUD Resource Operations

These operations cannot be handled by ldp-service alone because the OSLC 3.0 Link and Prefer headers for resource preview need to be included and handled. Here are the things that need to be handled:

OPTIONS, HEAD or GET response include Accept=application/x-oslc-compact+xml for resources that support resource preview
These same methods also include Link: ; rel="http://open-services.net/ns/core#Compact". The compact URL could be something like resource-url?compact or resource-url/compact, whatever the server wants. Note that LDPCs could also have compact representations.
GET with Accept=application/x-oslc-compact+xml returns Content-Type==application/x-oslc-compact+xml for resources with Compact representations
GET with Prefer: return=representation; include="http://open-services.net/ns/core#PreferCompact" includes the compact representation in line with the resource representation.

POST POST response should include a Link header for the Compact representation of the created resource: Link: ; rel="http://open-services.net/ns/core#Compact"; anchor="resource-url"

For POST operations to create new resource (perhaps using the creationFactor URI or an LDPC URI), the oslc-service could modify the request to use a specific kind of LDPC container. However, this implementation assumes:

all resources are created in LDPC BasicContainers (there is no attempt to use LDPC to implement specific OSLC Domain link types using DirectContainers).
ServiceProviderCatalog, ServiceProvider and Service are all LDPC BasicContainers that have additional rdf:types to support OSLC 2.0 discovery. This corresponds to IBM CE tools:
1. ServiceProviderCatalog - the root container referenced in the rootservices providerServices elements
2. ServiceProvider - corresponds to a Project Area
3. Service - a container for different types of resources owned by the tool and contained in a project area As a result, once these LDPCs are setup, the oslc-service can just forward the POST request to ldp-service.

 As you can see, these OSLC specific headers and entity response bodies are deeply coupled with LDP headers and content. This prevents oslc-service from using ldp-service in a simple express middleware chain of responsibility pattern.

The oslc-v3-sample implementation Sam and Steve did in Java implements a single set of REST services for OSLC V3. These services (re)implement whatever LDP capabilities are needed to support that server’s needs. For example, it only uses BasicContainers so there’s no implementation of DirectContainers.

Selective Properties

There’s a couple of ways this could be done, neither of which is a simple delegation to ldp-service.

GET the full resource, and prune the unselected properties
Do a specific query on the storage service to get only the properties requested #2 is preferred because this could be used internally for efficient creation of Compact resource representations. See Query Capability for additional ideas.

This could be implemented using storage services query method, and would simply not be routed to ldp-service.

OSLC Discovery

oslc-service extends LDP with:

LInk relations and response triples to indicate the types of resources that can be created in an LDPC
Link relations that provide resource shapes to constrain the creation of new member resources

OSLC 2.0 discovery resources can be implemented directly in oslc-service using storage service methods.

When an oslc-server server starts up, it reads a service config.json file that provides the server configuration information including:

{
    "scheme": "http",
    "host": "localhost",
    "port": 3000,
    "context": "/r",
    "storageImpl": "ldp-service-jena",
    "jenaURL": "http://localhost:3030/univ/",
    "services": path.resolve("./config/defaultServices.json"),
}

The config information could also reference a ServiceProviderCatalog that configures the (initial) services for the server. This file could be a JSON-LD file that defines the ServiceProviderCatalog, ServiceProviders and Services for the server, using OSLC 2.0 discover to specify the initial configuration of the repository.

A server could support updates to the service providers and services while it is running, possibly adding new ServiceProvider BasicContainers to organize different resources.

On startup, the oslc-server could read the config.json file, gets the ServiceProviderCatalog file, reads the JSON-LD and then ensures the service providers, services, and and all the creation factory LDPC URIs, etc. in the services document exists and creates them if they are not.

The service could merge the service provider information with the current server configuration. It does not need to delete any existing services that no longer appear in the ServiceProviderCatalog because that could loose data. Instead it is expected that an admin with sufficient access rights would use a console to manage old catalogs, an admin app could be provided that supported deleting or archiving old LDPCs, or they would simply be retained for compatibility with older apps.

The ServiceProviderCatalog, ServiceProvider and Service resources are LDPCs that are stored in persistent store with the properties specified in the config catalogs.

The IBM Jazz apps specify the services provide by a server in their rootservices resource, including the ServiceProviderCatalogs. This contains ServiceProvider entries for all the service providers, typically one for each project area.

This allows the services to be configured differently for each LDPC.

The @id’s in the catalogs are relative to the server. The full URL is added when the server is configured so that the stored resources have fully qualified URIs, without having to hard code that information in the spa.json or serviceProvider.json files.

The initial or default ServiceProviderCatalog could have one default service provider representing a generic LDPC that could be used for anything. Others could created dynamically by POSTing to create a new ServiceProvider LDPC. But the sample server could support the OSLC CM 3.0 and OSLC RM 2.1 vocabularies and shapes. Users could create their own folders in this root LDPC representing whatever organization of resources they want.

The default configuration should create a default LDPC the user can use for experimenting to create resources. POST can be used to create other LDPCs.

For example, RTC uses different LDPC/folder/creation factory for each change request type.

An oslc-server could use a single root LDPC for all ChangeRequests, and Requirements and allows any subtype to be created in that folder. Subtypes are discovered from the OSLC CM vocabulary. Servers could then organize the resources by function into sub-LDPCs instead of having type-specific LDPCs.

For example. RTC supports creation factories for:

ChangeRequest (these are all usages of ChangeRequest - another way of depicting subtypes using dcterms:type not rdf:type, The RTC admin UI allow creating and updating these usages and providing additional properties and editors)
- adoption
- task
- impediment
- defect
- planItem
- retrospective
- epic
- drafts
- buildtrackingitem
SelectionDialog
- default
- requirementsChangeRequest
- task
- defect
- planItem
CreationDialog
- task
- requirementsChangeRequest
- planItem
- defect, default (in the same oslc:Dialog as multiple usages)

Resource Preview

oslc-service GET needs to check for Accept=application/x-oslc-compact+xml and then construct a Compact resource for the request URI.

GET the resource in order to get the title, ideally reusing a storage service query (or selective properties subroutine)
look to see if the server has an icon for the resource based on its URI and an icons folder, and use a default icon if not
look to see if there's a smallPreview.ejs file for the resource based on its URI, construct a default reflectively if not including identifier, title, description, creator, created, modifiedBy, modified.
do the same thing for the large preview but include all properties including anchor tags for link properties with their title and URI.

This could be implemented using storage services query method, and would simply not be routed to ldp-service (no next() call).

Delegated Dialogs

Delegated selection and creation dialogs are only applicable to LDPCs. These will correspond to the Service BasicContainers.

Delegated dialog discovery is through Service creationDialog and selectionDialog properties and http://open-services.net/ns/core#creationDialog and http://open-services.net/ns/core#selectionDialog Link header relations.

These are examples of Link relations that are added by oslc-service after ldp-service has added its Link relations.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

High Level Design

oslc-service: An Express.js Middleware Component for OSLC services

Key features

Governance

High Level Design

Persistence Strategy

Configuration

Authentication

OSLC and LDP CRUD Resource Operations

Selective Properties

OSLC Discovery

Resource Preview

Delegated Dialogs

Attachments

Implementing the Change Management Domain

Using Resource Shapes

Creating Data Source Adapters

OSLC Query

OSLC Creation Factory

Resource Paging

Configuration Management

Tracked Resource Sets

Clone this wiki locally