Access to vhive-tuning S3 bucket denied.

[jingren1021]

I was trying to run Hyperparameter Tuning Halving benchmark and came across this issue where my access to vhive-tuning S3 bucket is denied.

jingren@hp-worker:~$ ~/vSwarm/tools/test-client/test-client -addr driver.default.192.168.1.240.sslip.io:80
2022/11/02 13:43:38 could not greet: rpc error: code = Unknown desc = Exception calling application: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

Kubectl logs as follow:

jingren@hp-worker:~$ kubectl logs -n default -c user-container -l serving.knative.dev/service=driver --tail=-1
INFO:root:Driver received a request
INFO:root:Driver uploading object with key 'dataset_key' to S3
ERROR:grpc._server:Exception calling application: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Traceback (most recent call last):
  File "/root/.local/lib/python3.8/site-packages/grpc/_server.py", line 444, in _call_behavior
    response_or_iterator = behavior(argument, context)
  File "./main.py", line 180, in SayHello
    event = self.handler_broker({}, {})
  File "./main.py", line 150, in handler_broker
    key = storage.put('dataset_key', dataset)
  File "/app/storage.py", line 72, in put
    s3object.put(Body=pickled)
  File "/root/.local/lib/python3.8/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/root/.local/lib/python3.8/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
  File "/root/.local/lib/python3.8/site-packages/botocore/client.py", line 388, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/root/.local/lib/python3.8/site-packages/botocore/client.py", line 708, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

I have exported AWS_ACCESS_KEY and AWS_SECRET_KEY environment variables and is able to run video analytics benchmark without issue. Is the public access to vhive-tuning S3 bucket denied by default or is it some issue with my setting on AWS account?
Thank you very much.

[ustiugov]

@jingren1021 thank you for reporting the issue. In fact, we should allow vSwarm users to configure their own buckets to avoid high bills to our side (except for the inputs).

@shyamjesal are the S3 buckets for ephemeral transfers all hardcoded, do you know?

[shyamjesal]

Yes, they are hardcoded currently.

[ustiugov]

@jingren1021 would you be able to contribute a fix? if you agree, we would try to help you find the LoC where the values are hardcoded

[jingren1021]

I have created a pull request on this. Please review #341 .
However, I am unable to create a new Docker image using Dockerfile (please see #334 ).

Thank you very much.

Edit: I was working on my forked repository and some unwanted changes were added to pull request #338 . I have undone the unwanted changes and created a new pull request. Please review #341 .

[jingren1021]

This has been fixed.