diff --git a/README.md b/README.md index c42d0bb..9541b13 100644 --- a/README.md +++ b/README.md @@ -26,8 +26,7 @@ Quick links ## Installation -To load the project in a Pharo image, or declare it as a dependency of your own -project follow this [instructions](docs/Installation.md). +To load the project in a Pharo image follow these [instructions](docs/how-to/how-to-load-in-pharo.md). ## Contributing diff --git a/docs/Installation.md b/docs/Installation.md deleted file mode 100644 index b561c40..0000000 --- a/docs/Installation.md +++ /dev/null @@ -1,60 +0,0 @@ -# Installation - -## Provided groups - -- `Core` will load the packages providing HTTPRequest building blocks to be used - in a deployed application -- `API Client` will load the packages providing API client building blocks to be - used in a deployed application -- `Service Discovery` will load the packages providing service discovery - strategies to be used in a deployed application -- `Deployment` will load all the packages needed in a deployed application - (Basically Core + API Client + Service Discovery) -- `Tests` will load the test cases -- `Dependent-SUnit-Extensions` will load the extensions to the SUnit framework -- `Tools` will load the extensions to the SUnit framework and development tools - (inspector and spotter extensions) -- `CI` is the group loaded in the continuous integration setup -- `Development` will load all the needed packages to develop and contribute to - the project - -## Basic Installation - -You can load **Superluminal** evaluating: - -```smalltalk -Metacello new - baseline: 'Superluminal'; - repository: 'github://ba-st/Superluminal:release-candidate'; - load. -``` - -> Change `release-candidate` to some released version if you want a pinned version - -## Using as dependency - -In order to include **Superluminal** as part of your project, you should -reference the package in your product baseline: - -```smalltalk -setUpDependencies: spec - - spec - baseline: 'Superluminal' - with: [ spec repository: 'github://ba-st/Superluminal:v{XX}' ]; - project: 'Superluminal-Deployment' copyFrom: 'Superluminal' - with: [ spec loads: 'Deployment' ]; -``` - -> Replace `{XX}` with the version you want to depend on - -```smalltalk -baseline: spec - - - spec - for: #common - do: [ self setUpDependencies: spec. - spec package: 'My-Package' - with: [ spec requires: #('Superluminal-Deployment') ] ] -``` diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..d3f9440 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,22 @@ +# Superluminal Documentation + +Superluminal provides building blocks for creating HTTP requests and API +clients, including: + +- Entity tags handling +- Caching +- Service Discovery + +To learn about the project, [install it](how-to/how-to-load-in-pharo.md) and +lookup for details in the reference docs: + +- [HTTP Requests](reference/HTTP-Requests.md) +- [API Clients](reference/API-Client.md) +- [Service Discovery](reference/Service-Discovery.md) + +--- + +To use the project as a dependency of your project, take a look at: + +- [How to use Superluminal as a dependency](how-to/how-to-use-as-dependency-in-pharo.md) +- [Baseline groups reference](reference/Baseline-groups.md) diff --git a/docs/how-to/how-to-load-in-pharo.md b/docs/how-to/how-to-load-in-pharo.md new file mode 100644 index 0000000..2189192 --- /dev/null +++ b/docs/how-to/how-to-load-in-pharo.md @@ -0,0 +1,35 @@ +# How to load Superluminal in a Pharo image + +## Using Metacello + +1. Download a [Pharo VM and image](https://pharo.org/download) +2. Open your Pharo image +3. Open a Playground +4. Evaluate: + + ```smalltalk + Metacello new + baseline: 'Superluminal'; + repository: 'github://ba-st/Superluminal:release-candidate'; + load: 'Development'. + ``` + +> Change `release-candidate` to some released version if you want a pinned version + +## Using Iceberg + +1. Download [pharo VM and image](https://pharo.org/download) +2. Open your Pharo image +3. Open Iceberg +4. Click the *Add* repository button +5. Select *Clone from github.com* and enter `ba-st` as owner name and `Superluminal` + as project name +6. Click *Ok* +7. Select the repository in the main Iceberg window +8. Open the contextual menu and select + *Metacello -> Install baseline of Superluminal ...* +9. Type `Development` and click *Ok* + +> After Iceberg cloned a repository, it will be checked-out at the default +> branch (in this case `release-candidate`). If you want to work on a different +> branch or commit perform the checkout before the baseline installation step. diff --git a/docs/how-to/how-to-use-as-dependency-in-pharo.md b/docs/how-to/how-to-use-as-dependency-in-pharo.md new file mode 100644 index 0000000..87101b6 --- /dev/null +++ b/docs/how-to/how-to-use-as-dependency-in-pharo.md @@ -0,0 +1,42 @@ +# How to use Superluminal as dependency in a Pharo product + +In order to include **Superluminal** as part of your project, you should reference +the package in your product baseline: + +1. Define the Superluminal repository and version to be used, and the [baseline groups](../reference/Baseline-groups.md) + you want to depend on (usually `Deployment`). + + If you're unsure about what to depend on use the *Dependency Analyzer* + tool to choose an appropriate group including the packages you need. + +2. Create a method like this one in the baseline class of your product: + + ```smalltalk + setUpDependencies: spec + + spec + baseline: 'Superluminal' + with: [ spec repository: 'github://github://ba-st/Superluminal:v{XX}' ]; + project: 'Superluminal-Deployment' + copyFrom: 'Superluminal' with: [ spec loads: 'Deployment' ] + ``` + + This will create `Superluminal-Deployment` as a valid target that can be used + as requirement in your own packages. + + > Replace `{XX}` with the version you want to depend on + +3. Use the new loading target as a requirement on your packages. For example: + + ```smalltalk + baseline: spec + + + spec + for: #pharo + do: [ + self setUpDependencies: spec. + spec + package: 'My-Package' + with: [ spec requires: #('Superluminal-Deployment') ] ] + ``` diff --git a/docs/reference/API-Client.md b/docs/reference/API-Client.md new file mode 100644 index 0000000..5bd8fb6 --- /dev/null +++ b/docs/reference/API-Client.md @@ -0,0 +1,140 @@ +# API Client + +When dealing with HTTP APIs, the `API Client` group in the baseline provides +abstractions to ease this kind of interaction. + +`RESTfulAPIClient` is the key abstraction for these features. Some of its +functionality is more useful when dealing with a RESTful API but is usable for +non-RESTful APIs as well. + +`RESTfulAPIClient` provides pooling of the underlying HTTP client, handling of +entity tags, and caching support. + +To create an API client you need to provide a block for creating a basic HTTP +client instance and the caching policy. + +For example: + +```smalltalk +RESTfulAPIClient + buildingHttpClientWith: [ ZnClient new ] + cachingIn: ExpiringCache onLocalMemory +``` + +will instantiate a client pooling ZnClient instances and using an in-memory cache. + +API clients support the following methods: + +- `getAt:configuredBy:withSuccessfulResponseDo:` will execute a GET against the + location in the first argument, configured by the second argument. If the + response is successful the last argument is evaluated with the response's body. +- `postAt:configuredBy:withSuccessfulResponseDo:` will execute a POST against the + location in the first argument, configured by the second argument. If the + response is successful the last argument is evaluated with the response's body. +- `putAt:configuredBy:withSuccessfulResponseDo:` will execute a PUT against the + location in the first argument, configured by the second argument. If + the response is successful, the last argument is evaluated with the response's + body, unless it is `204/No content`. +- `patchAt:configuredBy:withSuccessfulResponseDo:` will execute a PATCH against the + location in the first argument, configured by the second argument. If + the response is successful, the last argument is evaluated with the response's + body. +- `deleteAt:configuredBy:withSuccessfulResponseDo:` will execute a DELETE against + the location in the first argument, configured by the second argument. If the + response is successful the last argument is evaluated with the response's body. + +The configuration blocks follow the builder pattern defined [here](HTTP-Request.md). + +If the execution is unsuccessful an `HTTPClientError` exception is raised with +the corresponding response code. + +It also supports some convenience methods built on the previous ones: + +- `get:accepting:withSuccessfulResponseDo:` will execute a GET against the + location in the first argument, setting the `Accept` header according to the + second argument. If the response is successful the last argument is evaluated + with the response's body. +- `get:withSuccessfulResponseDo:` will execute a GET against the location in the + first argument without further configuration. If the response is successful + the last argument is evaluated with the response's body. +- `post:at:withSuccessfulResponseDo:` will execute a POST, whose body and + `Content-Type` is defined by the entity in the first argument, against the + second argument. If the response is successful the last argument is evaluated + with the response's body. +- `patch:at:withSuccessfulResponseDo:` will execute a PATCH, whose body and + `Content-Type` is defined by the entity in the first argument, against the + second argument. If the response is successful the last argument is evaluated + with the response's body. +- `put:at:` will execute a PUT, whose body and `Content-Type` is defined by the + entity in the first argument, against the second argument. If the response is + successful the last argument is evaluated with the response's body. +- `deleteAt:` will execute a DELETE against the location in the first argument. + If there's an entity tag associated with this location it will set the + `If-Match` header making the delete conditional. + +## Pooling + +Every time an API call is made, if a new authority and port is used +for the invocation the API client will create a connection pool. + +Currently, each pool created has at least one connection alive with +a maximum of 5 connections. + +On API client disposition all the connections in the pool are closed. + +## Entity tags handling + +The `ETag` HTTP response header is an identifier for a specific version of a +resource. It allows caches to be more efficient, and saves bandwidth, as a web +server does not need to send a full response if the content has not changed. On +the other side, if the content has changed, entity tags are useful to help prevent +simultaneous updates of a resource from overwriting each other ("mid-air collisions"). + +If the resource at a given URL changes, a new entity tag value must be generated. +Entity tags are therefore similar to fingerprints and might also be used for tracking +purposes by some servers. A comparison of them allows us to quickly determine +whether two representations of a resource are the same, but they might also be +set to persist indefinitely by a tracking server. + +`RESTfulAPIClient` takes advantage of entity tags, when present, in the following +scenarios: + +- When receiving a successful response including the `ETag` header, both the + entity tag value and the response body are cached in the client. +- When using `deleteAt:` or `patch:at:withSuccessfulResponseDo:` convenience + methods, if an entity tag was previously saved its value is used in an + `If-Match` header preventing deleting or patching a resource whose representation + has changed from the last time it was seen by the client. +- When using any of the `GET`-related methods, if an entity tag was previously + saved its value is used in an `If-None-Match` header. Giving the server the + chance to respond with `204/Not modified` and in that case, the previously + saved body is used as the response body. + +## Caching + +`RESTfulAPIClient` also provides a resource cache. It can be configured to use an +in-memory cache (inside the same image) or a shared cache using [Memcached](https://www.memcached.org/). + +To use an in-memory cache provide it to the API client doing: + +```smalltalk +ExpiringCache onLocalMemory +``` + +and, for a shared cache: + +```smalltalk +ExpiringCache onDistributedMemoryAt: serverList +``` + +where `serverList` is something like `{'127.0.0.1:11211'}` + +Both kinds of caches take into account the `Cache-Control` headers received in +the responses. When the API client receives any of the `GET`-related messages it +looks up in the cache if there's a non-expired resource cached for this location. +If it is, it will reuse that. In case there's no cached resource or the cached one +has expired it will proceed to execute the `GET`. + +Resources are cached when a successful `GET` response is received; and cleared when +any `POST`, `PUT`, `PATCH`, or `DELETE` method is executed against this location, +or when a cached resource is expired. diff --git a/docs/reference/Baseline-groups.md b/docs/reference/Baseline-groups.md new file mode 100644 index 0000000..ef0e6ab --- /dev/null +++ b/docs/reference/Baseline-groups.md @@ -0,0 +1,24 @@ +# Baseline Groups + +Superluminal includes the following groups in its Baseline that can be used as +loading targets: + +- `Core` will load the minimal required packages to support `HttpRequest` and + its building interface +- `API Client` will load the packages needed to support API clients, including: + - HTTP client pooling by authority + - ETag caching and automatic support for `If-None-Match` and `If-Match` headers, + and `Not modified` responses + - Caching support according to `Cache-Control` headers, both in-memory or using + memcached +- `Service Discovery` will load the packages needed to support service discovery + against a Consul agent +- `Deployment` will load all the packages needed in a deployed application, which + in this case correspond to `Core` + `API Client` + `Service Discovery` +- `Examples` will load a service discovery-enabled application +- `Tests` will load the test cases +- `Dependent-SUnit-Extensions` will load extensions to SUnit for testing API clients +- `CI` is the group loaded in the continuous integration setup, in this + particular case it is the same as `Tests` +- `Development` will load all the needed packages to develop and contribute to + the project diff --git a/docs/reference/HTTP-Request.md b/docs/reference/HTTP-Request.md new file mode 100644 index 0000000..cc7589f --- /dev/null +++ b/docs/reference/HTTP-Request.md @@ -0,0 +1,96 @@ +# HTTP Requests + +The library's base abstraction, provided by the `Core` baseline group, is `HttpRequest`. +It provides a builder-like interface to create HTTP requests that can be +later applied to an HTTP client (like `ZnClient` in Pharo). + +`HttpRequest` creates instances by using any of: + +- `get:configuredUsing:` +- `post:configuredUsing:` +- `delete:configuredUsing:` +- `patch:configuredUsing:` +- `put:configuredUsing:` + +where the first argument is anything convertible to an URL (it will receive the +`asUrl` message), and the second argument is a configuration closure providing +the builder interface. + +For example: + +```smalltalk +HttpRequest + get: 'https://api.example.com' + configuredUsing: [ :request | + request headers setBearerTokenTo: 'eJwdjdmOqlgARd' ]. +``` + +will produce the following + +```http +GET / HTTP/1.1 +Authorization: Bearer eJwdjdmOqlgARd +Host: api.example.com +``` + +## Configuring the request + +The builder instance injected into the configuration closure supports the +following messages: + +- `body` +- `headers` +- `queryString:` + +The `body` builder supports: + +- `contents:` receiving an `Entity` +- `contents:encodedAs:` will produce an entity including the contents in the + first argument with the media type in the second argument as its type. +- `formUrlEncoded:` produces a body of type `application/x-www-form-urlencoded` + with the fields declared in the argument block. +- `multiPart:` produces a body of type `multipart/form-data` with the fields and + files declared in the argument block. +- `json:` is syntactic sugar to easily produce a JSON body. The media type will + be `application/json`, and the argument will be converted to JSON by using `NeoJSONWriter`. + +The `headers` builder supports: + +- `set:to:` setting a header named as the first argument with the value in the + second one. +- `setAcceptTo:` will set the `Accept` header with the value in the argument. +- `setIfMatchTo:` and `setIfNoneMatchTo:` will receive an `ETag` and use it as the + corresponding value in `If-Match` or `If-None-Match` header. +- `setAuthorizationTo:` will set the value in the `Authorization` header +- `setBearerTokenTo:` is syntactic sugar over the previous one to easily set a + Bearer token directive. + +The `queryString:` builder supports: + +- `fieldNamed:pairedTo:` to add a query field with the provided name and value. + +The `formUrlEncoded:` builder supports: + +- `fieldNamed:pairedTo:` to add a form field with the provided name and value. + +The `multiPart:` builder supports: + +- `fieldNamed:pairedTo:` to add a form field with the provided name and value. +- `fieldNamed:attaching:` to add a file attachment with the provided name and content. + +## Applying the request + +Once the request is created, to execute it you need to apply it to some +compatible HTTP client by sending `applyOn:`. + +For example: + +```smalltalk +| request | +request := + HttpRequest + get: 'https://api.example.com' + configuredUsing: [ :request | + request headers setBearerTokenTo: 'eJwdjdmOqlgARd' ]. +request applyOn: ZnClient new +``` diff --git a/docs/reference/Service-Discovery.md b/docs/reference/Service-Discovery.md new file mode 100644 index 0000000..9f23d1e --- /dev/null +++ b/docs/reference/Service-Discovery.md @@ -0,0 +1,33 @@ +# Service Discovery + +When dealing with internal HTTP APIs, the `Service Discovery` group in the +baseline provides abstractions to ease the discovery of the hostname and port +attached to a service. + +These functions are performed by sending the message +`withLocationOfService:do:ifUnable:` to a service discovery client. Service +discovery clients can be chained, so if one client cannot discover a service +it will pass the query to the next one in the chain. + +The following service discovery clients are supported: + +- `NullServiceDiscoveryClient` will always fail when looking up services by their + name. In general, it's the last one in a discovery client chain. +- `FixedServiceDiscoveryClient` works with a fixed list of service locations. + It's useful for testing purposes or when transitioning from clients not using the + service discovery mechanics. It is chainable with another service discovery + client as a fallback. +- `ConsulAgentHttpAPIBasedDiscoveryClient` will use the [Consul HTTP API](https://www.consul.io/use-cases/service-discovery-and-health-checking) + to get a list of service locations. It uses the `Health` endpoint to obtain + the list of healthy services, then looks up the `Address` and `Port` provided + in the response. It is chainable with another service discovery client as a fallback. + Since it's querying an API it requires some configuration: + - `#agentLocation` must provide the location of the Consul agent + - `#retry` is optional and used to configure the retry policy in case the API + call fails. + + By default the client will retry up to 2 times when a `NetworkError` or + `HTTPError` happens during the API call, before failing and passing + control to the next client in the chain. + + The retry options can be anyone of those described [here](https://github.com/ba-st/Hyperspace/blob/release-candidate/docs/Resilience.md) diff --git a/source/BaselineOfSuperluminal/BaselineOfSuperluminal.class.st b/source/BaselineOfSuperluminal/BaselineOfSuperluminal.class.st index b38c1ed..d846a91 100644 --- a/source/BaselineOfSuperluminal/BaselineOfSuperluminal.class.st +++ b/source/BaselineOfSuperluminal/BaselineOfSuperluminal.class.st @@ -74,7 +74,7 @@ BaselineOfSuperluminal >> setUpDeploymentPackages: spec [ group: 'API Client' with: 'Superluminal-RESTfulAPI'; group: 'Deployment' with: 'Superluminal-RESTfulAPI'. spec - package: 'Superluminal-Service-Discovery' with: [ spec requires: 'Superluminal-Model' ]; + package: 'Superluminal-Service-Discovery' with: [ spec requires: 'Superluminal-RESTfulAPI' ]; group: 'Service Discovery' with: 'Superluminal-Service-Discovery'; group: 'Deployment' with: 'Superluminal-Service-Discovery' ] diff --git a/source/Superluminal-RESTfulAPI/RESTfulAPIClient.class.st b/source/Superluminal-RESTfulAPI/RESTfulAPIClient.class.st index 17a2629..5033744 100644 --- a/source/Superluminal-RESTfulAPI/RESTfulAPIClient.class.st +++ b/source/Superluminal-RESTfulAPI/RESTfulAPIClient.class.st @@ -61,7 +61,7 @@ RESTfulAPIClient >> clientPoolFor: aLocation [ ] ] -{ #category : #invoking } +{ #category : #'invoking - covenience' } RESTfulAPIClient >> deleteAt: aLocation [ ^ self @@ -74,7 +74,7 @@ RESTfulAPIClient >> deleteAt: aLocation [ withSuccessfulResponseDo: [ ] ] -{ #category : #invoking } +{ #category : #'invoking - covenience' } RESTfulAPIClient >> deleteAt: aLocation accepting: aMediaType withSuccessfulResponseDo: aBlock [ ^ self @@ -122,7 +122,7 @@ RESTfulAPIClient >> finalize [ ^ super finalize ] -{ #category : #invoking } +{ #category : #'invoking - covenience' } RESTfulAPIClient >> get: aLocation accepting: aMediaType withSuccessfulResponseDo: aBlock [ ^ self @@ -150,7 +150,7 @@ RESTfulAPIClient >> get: aLocation executing: httpRequest [ ] ] -{ #category : #invoking } +{ #category : #'invoking - covenience' } RESTfulAPIClient >> get: aLocation withSuccessfulResponseDo: aMonadicBlock [ ^ self getAt: aLocation configuredBy: [ ] withSuccessfulResponseDo: aMonadicBlock @@ -203,34 +203,41 @@ RESTfulAPIClient >> normalize: aLocation [ ^aLocation asString ] -{ #category : #invoking } +{ #category : #'invoking - covenience' } RESTfulAPIClient >> patch: anEntity at: aLocation withSuccessfulResponseDo: aBlock [ ^ self - handleExceptionsDuring: [ - | httpRequest response | - httpRequest := HttpRequest - patch: aLocation - configuredUsing: [ :request | - | command | - command := request body contents: anEntity. - self - withCachedETagAt: aLocation - do: [ :entityTag | command := command + ( request headers setIfMatchTo: entityTag asString ) ]. - command - ]. - self - withHttpClientFor: aLocation - do: [ :httpClient | response := httpRequest applyOn: httpClient ]. - response isSuccess - ifTrue: [ expiringCache clearResourceAt: aLocation. - aBlock value: ( self tryToCacheContentsOf: response basedOn: aLocation ) - ] - ifFalse: [ self signalCannotCompleteUpdateErrorBasedOn: response ] - ] + patchAt: aLocation + configuredBy: [ :request | + | command | + + command := request body contents: anEntity. + self withCachedETagAt: aLocation + do: [ :entityTag | command := command + ( request headers setIfMatchTo: entityTag asString ) ]. + command + ] + withSuccessfulResponseDo: aBlock ] { #category : #invoking } +RESTfulAPIClient >> patchAt: aLocation configuredBy: aRequestBuildingBlock withSuccessfulResponseDo: aSuccessBlock [ + + ^ self handleExceptionsDuring: [ + | httpRequest response | + + httpRequest := HttpRequest patch: aLocation configuredUsing: aRequestBuildingBlock. + self withHttpClientFor: aLocation + do: [ :httpClient | response := httpRequest applyOn: httpClient ]. + response isSuccess + ifTrue: [ + expiringCache clearResourceAt: aLocation. + aSuccessBlock value: ( self tryToCacheContentsOf: response basedOn: aLocation ) + ] + ifFalse: [ self signalCannotCompleteUpdateErrorBasedOn: response ] + ] +] + +{ #category : #'invoking - covenience' } RESTfulAPIClient >> post: anEntity at: aLocation withSuccessfulResponseDo: aBlock [ ^ self @@ -273,7 +280,7 @@ RESTfulAPIClient >> processPutResponse: response at: aLocation whenSuccessfulDo: ^ self signalCannotCompleteUpdateErrorBasedOn: response ] -{ #category : #invoking } +{ #category : #'invoking - covenience' } RESTfulAPIClient >> put: anEntity at: aLocation [ ^ self