Skip to content

Latest commit

 

History

History
123 lines (89 loc) · 4.95 KB

13.SQSDirectSource.md

File metadata and controls

123 lines (89 loc) · 4.95 KB

Adding AWS SQS Source as Direct Source to your Application

Prerequisites

  • Create a queue using Simple Queue Service in AWS. Note the URL for your queue.
  • Cluster administrator should install AWS SQS Controller on your cluster
  • You will need your aws credentials to configure as a secret
  • You have a Knative service deployed. Add a knative service running kn service create msgtxr-sl --image=image-registry.openshift-image-registry.svc:5000/kn-demo/msgtxr -l app.openshift.io/runtime=quarkus --env format=none

Add a secret

Create a file with aws credentials

% cat credentials 
[default]
aws_access_key_id = <awsaccesskey>
aws_secret_access_key = <awssecretkey>

Create a secret referencing this file

oc create secret generic aws-credentials --from-file=credentials=./credentials

Add AWS SQS Source

We will now create a source that reads messages posted to your AWS SQS Queue and sinks those messages to your knative service.

AwsSqsSource is a custom resource definition added when the administrator installed the controller. We will create a custom resource for our application. The controller manages this custom resource.

Let us understand the AwsSqsSource custom resource shown below. This reads the data from SQS queue on AWS and pushes it to the sink. In our case we are configuring sink to be our knative service. Also note that this source references the secret created above.

% cat awssqs/awssqs-source-direct.yaml
# Replace the following before applying this file:
#   QUEUE_URL: Replace with the AWS SQS queue.

apiVersion: sources.knative.dev/v1alpha1
kind: AwsSqsSource
metadata:
  name: awssqs-sample-source
spec:
  awsCredsSecret:
    name: aws-credentials
    key: credentials
  queueUrl: QUEUE_URL
  sink:
    apiVersion: serving.knative.dev/v1
    kind: Service
    name: msgtxr-sl

Set the value of an environment variable to your SQS URL

export QUEUE_URL=<yourSQSURL>

Let us create the source by replacing the value of QUEUE_URL running

sed -e "s|QUEUE_URL|$QUEUE_URL|g" awssqs/awssqs-source-direct.yaml | oc create -f -

and notice

awssqssource.sources.knative.dev/awssqs-sample-source created

You should see the awssqssource pod running.

% oc get po 
NAME                                                 READY   STATUS    RESTARTS   AGE
awssqs-awssqs-sample-source-k9rcm-7d48dbb7f6-l2mld   1/1     Running   0          116s

It will show up on your developer console as below:

topology

Post Messages and Test

Now post a message to the queue running the following command from command line. Alternately you can also post a message from AWS Console

aws sqs send-message --queue-url=$QUEUE_URL --message-body="hello world"

Notice the knative service pod scales up and the logs show the following output.

__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2020-06-24 20:34:54,779 INFO  [io.quarkus] (main) getting-started 1.0-SNAPSHOT (powered by Quarkus 1.3.2.Final) started in 0.017s. Listening on: http://0.0.0.0:8080
2020-06-24 20:34:54,779 INFO  [io.quarkus] (main) Profile prod activated. 
2020-06-24 20:34:54,779 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy]
20:34:55.864 IN {"Attributes":{"SentTimestamp":"1593030891645"},"Body":"hello world","MD5OfBody":"5eb63bbbe01eeed093cb22bb8f5acdc3","MD5OfMessageAttributes":null,"MessageAttributes":null,"MessageId":"149e8578-ccc5-4e80-911e-d898dbf8905d","ReceiptHandle":"AQEBdEhLPsrD6TdESml95/OVJYeFPUvcfEsELxqaspiHN/ZlrDehmJdw9gQeycxud4KwTBXdelTmSPPR1Jfjn/nTVzvtlEDubGeH1mrPAzb4R4Du1n9FEZrGMJKGAwpp5TNaa7ynaoEJm7LFk3x9X0LQAucMZkolBL0NpW1zud9ouASb11Iqv/OEb087AnrhtKu6StCXqX9sxkKL2scNUOhIExg9EKtX9Gr77VrX+ynSo2ZPfBCDlfDtQiQ1MTV6bY207/zej2mgjxUXVHqTfBWY/0wADwkM6W5niKaQCW59o92YSC4tOdJVuoZRjpamT79WLPjdf6N6hR6uAM3230VBzxeDTKzFRwb6x0J3++Lc2jFUHlJ5W4rPd7CmB9+LHHJMYM9JzNhGbR9eaVeXqPe6mg=="} OUT {"Attributes":{"SentTimestamp":"1593030891645"},"Body":"hello world","MD5OfBody":"5eb63bbbe01eeed093cb22bb8f5acdc3","MD5OfMessageAttributes":null,"MessageAttributes":null,"MessageId":"149e8578-ccc5-4e80-911e-d898dbf8905d","ReceiptHandle":"AQEBdEhLPsrD6TdESml95/OVJYeFPUvcfEsELxqaspiHN/ZlrDehmJdw9gQeycxud4KwTBXdelTmSPPR1Jfjn/nTVzvtlEDubGeH1mrPAzb4R4Du1n9FEZrGMJKGAwpp5TNaa7ynaoEJm7LFk3x9X0LQAucMZkolBL0NpW1zud9ouASb11Iqv/OEb087AnrhtKu6StCXqX9sxkKL2scNUOhIExg9EKtX9Gr77VrX+ynSo2ZPfBCDlfDtQiQ1MTV6bY207/zej2mgjxUXVHqTfBWY/0wADwkM6W5niKaQCW59o92YSC4tOdJVuoZRjpamT79WLPjdf6N6hR6uAM3230VBzxeDTKzFRwb6x0J3++Lc2jFUHlJ5W4rPd7CmB9+LHHJMYM9JzNhGbR9eaVeXqPe6mg=="}

Post a few more messages and test.

Conclusion

In this chapter we have learnt to configure AWS SQS as a source for your knative service

Cleanup

Delete AWS SQS Source

oc delete -f awssqs/awssqs-source-direct.yaml 
awssqssource.sources.knative.dev

Delete Secret

oc delete secret/aws-credentials