-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore: Update IAM and Terrapi URLs and platform scopes
- Loading branch information
1 parent
bea4a72
commit d2a4630
Showing
2 changed files
with
257 additions
and
4 deletions.
There are no files selected for viewing
252 changes: 252 additions & 0 deletions
252
notebooks/Discover_and_access_client_app_workspaces.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,252 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Discover and access client app workspaces\n", | ||
| "\n", | ||
| "This notebook is intended for development and testing purpose only. It shall allow you to discover the available workspaces and access them on behalf of a client application.\n", | ||
| "\n", | ||
| "The use case is an application that needs to access TerrAPI in order to upload data to a workspace from a client application.\n", | ||
| "\n", | ||
| "In this notebook, we will:\n", | ||
| "- Authenticate with the API as the client app in order to get an access token\n", | ||
| "- Get the user information from TerrAPI\n", | ||
| "- List the workspaces of the user\n", | ||
| "- Select a workspace and access it\n", | ||
| "- Upload a file to the workspace\n", | ||
| "- Get a download link for the file (signed url)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "#################################################\n", | ||
| "# First Let's set up the environment\n", | ||
| "iam_url = 'https://iam.terradue.com/realms/master'\n", | ||
| "terrapi_url = 'https://api.terradue.com/core'\n", | ||
| "platform_scopes = ['gep'] # list of platform scopes. See https://api.bios-dev.terradue.com/core/docs/platforms/ for more information\n", | ||
| "\n", | ||
| "# Set localdev to True if you are running the notebook to services running on your local machine (self-signed certificates)\n", | ||
| "localdev = False\n", | ||
| "#################################################\n", | ||
| "\n", | ||
| "import sys, os\n", | ||
| "import pprint\n", | ||
| "sys.path.append('../')\n", | ||
| "import openapi_client\n", | ||
| "from openapi_client.rest import ApiException\n", | ||
| "\n", | ||
| "configuration = openapi_client.Configuration(\n", | ||
| " host = terrapi_url\n", | ||
| ")\n", | ||
| "configuration.verify_ssl = not(localdev)\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Authentication\n", | ||
| "\n", | ||
| "The following code will authenticate with the TerrAPI and get an access token. You can find more information in the [TerrAPI documentation](https://docs.terrapia.io/docs/authorization)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from IPython.display import JSON\n", | ||
| "from authlib.jose import jwt\n", | ||
| "import requests\n", | ||
| "\n", | ||
| "\n", | ||
| "pp = pprint.PrettyPrinter(indent=4, width=80, compact=True)\n", | ||
| "\n", | ||
| "# First request an access token from keycloak using the client credentials flow\n", | ||
| "# and use the access token to create an authenticated client\n", | ||
| "# the client will automatically refresh the token when it expires\n", | ||
| "\n", | ||
| "from authlib.oidc.discovery import well_known, OpenIDProviderMetadata\n", | ||
| "t2_oidc_dicovery_url = well_known.get_well_known_url(iam_url, external=True)\n", | ||
| "t2_oidc_dicovery = OpenIDProviderMetadata(requests.get(t2_oidc_dicovery_url, verify=not(localdev)).json())\n", | ||
| "\n", | ||
| "# We will use the public client of Terradue IAM\n", | ||
| "client_id = 'app_client1'\n", | ||
| "client_secret = 'app_client1_secret'\n", | ||
| "# Scopes requested (the platform related scopes must be set accordingly)\n", | ||
| "scopes = 'openid email profile offline_access' + ' '.join([' {}'.format(s) for s in platform_scopes])\n", | ||
| "\n", | ||
| "# using requests implementation\n", | ||
| "from authlib.integrations.requests_client import OAuth2Session\n", | ||
| "from authlib.common.security import generate_token\n", | ||
| "client = OAuth2Session(client_id, client_secret, scope=scopes)\n", | ||
| "redirect_uri='http://localhost:8888/terrapi/'\n", | ||
| "\n", | ||
| "authorization_endpoint = t2_oidc_dicovery['authorization_endpoint']\n", | ||
| "token_endpoint = t2_oidc_dicovery['token_endpoint']\n", | ||
| "token = client.fetch_token(token_endpoint, grant_type='client_credentials')\n", | ||
| "os.environ['ACCESS_TOKEN'] = token['access_token']\n", | ||
| "os.environ['REFRESH_TOKEN'] = token['refresh_token']\n", | ||
| "pp.pprint(token)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Now you usually have an **ACCESS TOKEN** that we will use to make requests to the API. The access token is actually a JWT token that contains the user's information. The token is signed by the server and can be verified by the server. The token is also encrypted so that the user's information is not visible to anyone else.\n", | ||
| "Let's use it to get the user's information." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "configuration.access_token = os.environ[\"ACCESS_TOKEN\"]\n", | ||
| "# Enter a context with an instance of the API client\n", | ||
| "with openapi_client.ApiClient(configuration) as api_client:\n", | ||
| " # Create an instance of the API class\n", | ||
| " api_instance = openapi_client.UserApi(api_client)\n", | ||
| " job_id = 'job_id_example' # str | \n", | ||
| "\n", | ||
| " try:\n", | ||
| " # Get the user information\n", | ||
| " userinfo = api_instance.v2_user_info_get()\n", | ||
| " print(\"The response of UserAPI:\\n\")\n", | ||
| " pp.pprint(userinfo)\n", | ||
| " except ApiException as e:\n", | ||
| " print(\"Exception when calling UserAPI: %s\\n\" % e)\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## List workspaces\n", | ||
| "\n", | ||
| "We will use the access token to get the available workspaces. We will then select a workspace and access it." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "configuration.access_token = os.environ[\"ACCESS_TOKEN\"]\n", | ||
| "# Enter a context with an instance of the API client\n", | ||
| "with openapi_client.ApiClient(configuration) as api_client:\n", | ||
| " # Create an instance of the API class\n", | ||
| " api_instance = openapi_client.StorageApi(api_client)\n", | ||
| "\n", | ||
| " try:\n", | ||
| " # Get the storage information\n", | ||
| " workspaces = api_instance.get_workspaces()\n", | ||
| " except ApiException as e:\n", | ||
| " print(\"Exception when calling StorageApi: %s\\n\" % e)\n", | ||
| "\n", | ||
| "\n", | ||
| "# We select the service workspace of the current platform\n", | ||
| "# Usually the default workspace is the one with the naming convention <platform>-<username>-private-workspace\n", | ||
| "workspace = next(filter(lambda x:x.name == 'gep-processing-results-a2s-service-workspace',workspaces))\n", | ||
| "\n", | ||
| "pp.pprint(workspace)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Get a Security Token to access the workspace\n", | ||
| "\n", | ||
| "In order to access the workspace via the storage provider, we need to get a security token. The security token is a set of credentials generated signed by the API and that can be verified by the storage provider." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "configuration.access_token = os.environ[\"ACCESS_TOKEN\"]\n", | ||
| "# Enter a context with an instance of the API client\n", | ||
| "with openapi_client.ApiClient(configuration) as api_client:\n", | ||
| " # Create an instance of the API class\n", | ||
| " api_instance = openapi_client.StorageApi(api_client)\n", | ||
| "\n", | ||
| " try:\n", | ||
| " # Request a security token to access the workspace\n", | ||
| " token = api_instance.get_storage_sts(workspace.name)\n", | ||
| " except ApiException as e:\n", | ||
| " print(\"Exception when calling StorageApi: %s\\n\" % e)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Access the workspace using the token\n", | ||
| "\n", | ||
| "We will use the security token to access the workspace and upload a file. All the information about the storage access and credentials are contained in the security token.\n", | ||
| "In this case, it is a S3 storage provider, so we will use the `boto3` library to access the workspace using the parameters in the security token." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import boto3\n", | ||
| "import boto3.session\n", | ||
| "\n", | ||
| "## Create a S3 session with the service URL and the security token\n", | ||
| "session = boto3.session.Session(aws_access_key_id=token.credentials.access_key, \n", | ||
| " aws_secret_access_key=token.credentials.secret_key, \n", | ||
| " aws_session_token=token.credentials.token,\n", | ||
| " region_name=token.storage.properties.__contains__('region') and token.storage.properties['region'][0] or 'us-east-1')\n", | ||
| "\n", | ||
| "## Create a S3 client with the service URL and the security token\n", | ||
| "s3 = session.client('s3', endpoint_url=token.storage.properties['service_url'][0], verify=not(localdev))\n", | ||
| "\n", | ||
| "pp.pprint(token.storage.service_uri)\n", | ||
| "pp.pprint(token.storage.properties)\n", | ||
| "pp.pprint(token.credentials.token)\n", | ||
| "\n", | ||
| "## List the content of the workspace\n", | ||
| "response = s3.list_objects_v2(Bucket=token.storage.properties['bucket'][0], Prefix=token.storage.properties['key'][0])\n", | ||
| "pp.pprint(response)" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": ".venv", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.11.6" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters