From 54a8c8e29e6f791d344380c44cb5e4977b1ce885 Mon Sep 17 00:00:00 2001 From: Jason Stidd Date: Thu, 18 May 2023 13:01:33 -0600 Subject: [PATCH] add readme to globus-compute-endpoint (#128) * add readme to globus-compute-endpoint * bump chart version --- charts/globus-compute/Chart.yaml | 2 +- charts/globus-compute/README.md | 116 +++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 1 deletion(-) create mode 100644 charts/globus-compute/README.md diff --git a/charts/globus-compute/Chart.yaml b/charts/globus-compute/Chart.yaml index 5239de4..492748f 100644 --- a/charts/globus-compute/Chart.yaml +++ b/charts/globus-compute/Chart.yaml @@ -6,7 +6,7 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. -version: 0.1.4 +version: 0.1.5 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. diff --git a/charts/globus-compute/README.md b/charts/globus-compute/README.md new file mode 100644 index 0000000..afb7d12 --- /dev/null +++ b/charts/globus-compute/README.md @@ -0,0 +1,116 @@ +# Kubernetes Endpoint +This chart will deploy a functioning Kubernetes endpoint into your SLATE cluster. It +will launch workers with a specified container image into a namespace. + +## How to Use +There are two required values to specify in the `values.yaml` file: +`endpointUUID` and authentication. The `endpointUUID` is easy: + +``` +endpointUUID: +``` + +The UUID is of your choosing, but must be available. In particular, if the +UUID you choose has already been taken, the endpoint will fail to register. +One method to generate a UUID is with the `uuid` command line tool: + +```shell +$ sudo apt install uuid +... +$ uuid # will generate a version 1 UUID. +e22be136-b3eb-11ed-8611-5b7bc2d2f962 +``` + +Alternatively, Python has the builtin `uuid` module: +```python +>>> import uuid +>>> uuid.uuid4() +UUID('ea0cab7e-b3eb-11ed-ae8b-719a5541eacb') +``` + +Getting the authentication setup is slightly more involved. Under the hood, +the Globus Compute Endpoint uses the Globus Compute SDK for communication with the web services, +which requires an authenticated user for most API routes. The Globus Compute SDK can +use either client credentials or user credentials. This README shows how to implement the client credentials. + +#### Client Credentials +The Globus Compute SDK supports use of Globus Auth Client Credentials. In practice, +that means exporting two variables into the endpoint's environment: + +* `FUNCX_SDK_CLIENT_ID` +* `FUNCX_SDK_CLIENT_SECRET` + +These variables may be generated by following the steps in the [Registering an +Application](https://docs.globus.org/api/auth/developer-guide/#register-app) +section on the [Globus Auth Developer's +Guide](https://docs.globus.org/api/auth/developer-guide/). + +Outside of this chart, use of client credentials is also documented for [normal +Globus Compute SDK +usage](https://funcx.readthedocs.io/en/latest/sdk.html#client-credentials-with-globus-compute-clients). + +Add these variables to a secret object in Kubernetes. For example, to put them +into a Kubernetes store named `my-secrets`, you could create a temporary env file +and load them: + +``` +$ (umask 077; touch client_creds.env) # create with 0600 (-rw-------) perms +$ cat > client_creds.env +FUNCX_SDK_CLIENT_ID=11111111-2222-4444-8888-000000000000 +FUNCX_SDK_CLIENT_SECRET=yoursecret +^D +$ slate secret create my-secrets --grouop --cluster --from-env-file ./client_creds.env +``` + +Then, specify the secret name in the configuration file, and tell the chart to use +the client credentials: +``` +secrets: my-secrets +useClientCredentials: true +``` + +## Install the Globus Compute Endpoint + +Download the configuration file: + +```sehll script +slate app get-conf globus-compute-endpoint > your-config.yaml +``` + +Update the configuration file with the parameters covered above, and install the application with: + +```shell script +slate app install globus-compute-endpoint --group --cluster --config your-config.yaml +``` + +## Values +The deployment is configured via values.yaml file. + +| Value | Description | Default | +|-------| ----------- | ------- | +| Globus ComputeServiceAddress | URL for the FuncX Webservice. | https://api.funcx.org | +| image.repository | Docker image repository | funcx/kube-endpoint | +| image.tag | Tag name for the endpoint image | endpoint_helm | +| image.pullPolicy | Pod pull policy for the endpoint image | Always | +| workerDebug | Log additional information in the worker logs | False | +| workerImage | Docker image to run in the worker pods | python:3.6-buster | +| workerInit | Command to execute on worker before strating uip | pip install parsl==0.9.0;pip install --force-reinstall globus-compute-sdk>=2.0.0 | +| workerNamespace | Kubernetes namespace to launch worker pods into | default | +| workingDir | Directory inside the container where log files are to be stored | /tmp/worker_logs | +| rbacEnabled | Create service account and roles? | true | +| initMem | Initial memory for worker pod | 2000Mi | +| maxMem| Maximum allowed memory for worker pod | 16000Mi | +| initCPU | Initial CPUs to allocate to worker pod | 1 | +| maxCPU | Maximum CPUs to allocate to worker pod | 2 | +| maxBlocks | Maximum number of worker pods to spawn | 100 | +| maxWorkersPerPod | How many workers will be scheduled in each pod | 1 | +| taskTTLSeconds | (Optional) If set, will stop tasks that run longer than this value, in (fractional) seconds. Example: 1.5 | | +| endpointName | (Optional) Specify a name for registration with the funcX web services | The release name (Release.Name) | +| endpointDisplayName | (Optional) Specify a display name for registration with the funcX web services | The endpoint name (endpointName) or the release name (Release.Name) | +| endpointUUID | (Required) Specify a UUID for this endpoint. | | +| endpointCLIargs | Any additional command line arguments to give to the `globus-compute-endpoint` executable | | +| maxIdleTime | The maximum time to maintain an idle worker. After this time the SimpleStrategy will terminate the idle worker. | 3600 | +| imagePullSecret | The K8s secret to use to deploy worker images. This can refer to an ECR secret. | | +| secrets | Kubernetes secret object in which to find client credential environment variables | | +| useClientCredentials | Whether to use _client_ credentials | false | +| useUserCredentials | Whether to use _user_ credentials (i.e., `storage.db`) | false |