Home

Execution of applications through Chronos

Create a docker image for your job including oneclient

1. Create a new repository on github

   Write the following Dockerfile

   FROM ubuntu:16.04
   MAINTAINER Marica Antonacci 
   LABEL description="Container image to run R scripts"
   RUN apt-get update -y
   RUN apt-get install software-properties-common -y
   RUN apt-add-repository ppa:ansible/ansible
   RUN apt-get update && \
    apt-get install -y \
        ansible wget
   RUN ansible-galaxy install maricaantonacci.r && \
       ansible-galaxy install indigo-dc.oneclient && \
       ansible-playbook /etc/ansible/roles/maricaantonacci.r/tests/base.yml && \
       ansible-playbook /etc/ansible/roles/indigo-dc.oneclient/tests/test.yml
   RUN apt-get clean \
       && rm -rf /var/lib/apt/lists/*
   ENV AUTHENTICATION=token
   ENV NO_CHECK_CERTIFICATE=true
   

2. Create the automated build in Docker Hub

   Trigger the build from the tab 'Build Settings'

Prepare the TOSCA template for your job

1. Prepare the script to run your job

   See an example here:

2. Prepare the TOSCA template

   See an example here

Submit your job to INDIGO orchestrator

1. Get a IAM access token. You can use this script
   Note: the script needs jq installed on the machine

      
   $ export IAM_CLIENT_ID=xxxx
   $ export IAM_CLIENT_SECRET=xxxx 
   $ ./bin/get-iam-access-token.sh
   

   The script returns the IAM access token.

   Export the environment variable IAM_ACCESS_TOKEN:

      
   $ export IAM_ACCESS_TOKEN=xxxx
   

2. Download and run the Orchestrator CLI orchent:

   wget https://github.com/indigo-dc/orchent/releases/download/0.4.0/orchent_container_0.4.0.tar
   
   docker load --input orchent_container_0.4.0.tar
   Output:
   5f649f8aedb9: Loading layer [==================================================>] 8.041 MB/8.041 MB
   5346a4243f43: Loading layer [==================================================>] 446.5 kB/446.5 kB
   
   

   Then you can configure the following alias:

   export ORCHENT_TOKEN=$IAM_ACCESS_TOKEN
   export ORCHENT_URL="https://orchestrator01-indigo.cloud.ba.infn.it:443/orchestrator" 
   alias orchent='docker run --rm -e ORCHENT_TOKEN=$ORCHENT_TOKEN -e ORCHENT_URL=$ORCHENT_URL -v $PWD:/data orchent:0.4.0'
   

   Test the orchent:

   # orchent --help
   

3. Submit your template

   orchent depcreate /data/emso_job.yaml '{"input_onedata_token": "******","output_onedata_token": "****","input_onedata_providers": "oneprovider2.cloud.ba.infn.it","output_onedata_providers": "oneprovider2.cloud.ba.infn.it","input_onedata_space": "demo","output_onedata_space": "demo","input_path": "test/input","output_path": "test/output","output_filenames": "cum_e.out,Rplots.pdf","cpus": "1","mem": "1024 MB","docker_image": "marica/test","N": "10"}'
   

   The output is like this:

   Deployment [654f99a6-135f-4048-8964-fd3f01ac7a68]:
     status: CREATE_IN_PROGRESS
     creation time: 2017-06-17T15:10+0000
     update time: 2017-06-17T15:10+0000
     callback: 
     status reason: 
     outputs: 
     {}
     links:
       self [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/654f99a6-135f-4048-8964-fd3f01ac7a68]
       resources [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/654f99a6-135f-4048-8964-fd3f01ac7a68/resources]
       template [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/654f99a6-135f-4048-8964-fd3f01ac7a68/template]
   

4. Monitor your job execution

   orchent depshow < UUID >
   

Deployment of long running service through Marathon

1. Prepare the TOSCA template See an example here

2. Submit your template

      
   orchent depcreate /data/mysql.yaml '{}'
   

   Output:

   Deployment [21a16b6d-6729-4502-8e8c-a47d9049988b]:
    status: CREATE_IN_PROGRESS
    creation time: 2017-06-19T19:59+0000
    update time: 2017-06-19T19:59+0000
    callback: 
    status reason: 
    outputs: 
    {}
    links:
      self [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/21a16b6d-6729-4502-8e8c-a47d9049988b]
      resources [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/21a16b6d-6729-4502-8e8c-a47d9049988b/resources]
      template [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/21a16b6d-6729-4502-8e8c-a47d9049988b/template]
   

3. Monitor the deployment

   orchent depshow < UUID >
   

   When the deployment is finalized, you will see the following output:

   Deployment [21a16b6d-6729-4502-8e8c-a47d9049988b]:
    status: CREATE_COMPLETE
    creation time: 2017-06-19T19:59+0000
    update time: 2017-06-19T19:59+0000
    callback: 
    status reason: 
    outputs: 
    {}
    links:
      self [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/21a16b6d-6729-4502-8e8c-a47d9049988b]
      resources [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/21a16b6d-6729-4502-8e8c-a47d9049988b/resources]
      template [https://orchestrator01-indigo.cloud.ba.infn.it/orchestrator/deployments/21a16b6d-6729-4502-8e8c-a47d9049988b/template]
   

4. Access the deployed service through the cluster VIP and the service port specified in the template.