Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop privileges in container image so it can run securely on Kubernetes #2

Closed
wants to merge 1 commit into from

Conversation

pcktdmp
Copy link

@pcktdmp pcktdmp commented Apr 21, 2020

Compute clusters have the tendency of sharing resources with mixed workloads where Folding at Home could be one of them.

To guarantee high(er) integrity of the compute clusters security and mainly integrity of the data, workload (containers) should not run in privileged mode.

In the case of Kubernetes this means that upfront inside the resource specification it needs to be known under which UID and GID the process will run.

The VOLUME statement inside the container could be omitted if attached during runtime of the docker command instead of defining this in the Dockerfile to make the image portable (across schedulers).

Dropping privileges can also be done by creating an entrypoint.sh wrapper script, but that makes the solution less elegant and also rules out Kubernetes support with a (stricter) securityContext set since the container starts as root, but later on drops it out of Kubernetes-es line of sight.

…yContext active which prohibites running privileged containers
@beberg
Copy link
Contributor

beberg commented Apr 21, 2020

Mounting a persistent storage volume into /fah is mandatory, not optional. There should be one for each container and it should have the config.xml preloaded - see "Running on a Cluster" in the README.

The container can also be run under any UID, either by Docker (from README):

docker run --gpus all --name fah0 -d --user "$(id -u):$(id -g)" \
  --volume $HOME/fah:/fah foldingathome/fah-gpu:latest

Or in K8s securityContext or other container orchestrators.

@jcoffland
Copy link
Member

The .deb package already does this. It adds a user fahclient and the client software drops privileges.

@beberg
Copy link
Contributor

beberg commented Apr 22, 2020

The .deb package already does this. It adds a user fahclient and the client software drops privileges.

There are no services in containers (in general), so the fahclient user is not what we're talking about in this case. The user a container runs as is controlled by docker/K8s/etc. The .deb is only being used to get the FAHClient binary installed and trigger dependencies.

@jcoffland
Copy link
Member

Ah, right it just runs the client process in the container. You should be able to add the option <run-as v="fahclient"/> in the config.xml or --run-as=fahclient on the command line.

@beberg
Copy link
Contributor

beberg commented Apr 22, 2020

Ah, right it just runs the client process in the container. You should be able to add the option <run-as v="fahclient"/> in the config.xml or --run-as=fahclient on the command line.

It's the --user "$(id -u):$(id -g)" part of the docker command. or in the K8s securityContext. The fahclient just kinda tagged along with the .deb.

@beberg beberg closed this Apr 22, 2020
@pcktdmp
Copy link
Author

pcktdmp commented Apr 22, 2020

Mounting a persistent storage volume into /fah is mandatory, not optional. There should be one for each container and it should have the config.xml preloaded - see "Running on a Cluster" in the README.

The container can also be run under any UID, either by Docker (from README):

docker run --gpus all --name fah0 -d --user "$(id -u):$(id -g)" \
  --volume $HOME/fah:/fah foldingathome/fah-gpu:latest

Or in K8s securityContext or other container orchestrators.

With "omitted" I meant that you could achieve the same effect with docker volume create data1 && docker run -ti -v data1:/data alpine df -h.

In the above example it is assumed that the running uid and gid has permissions to a volume attached inside the container, which typically not the case when it's a fresh volume(claim) inside Kubernetes.

$ docker volume create data1
$ docker run --user "9999:9999" -v data1:/fah -ti foldingathome/fah-gpu:latest
20:18:31:ERROR:Exception: Failed to open 'log.txt': Failed to open 'log.txt': Permission denied: iostream error: Permission denied

Hence the VOLUME statement inside Dockerfile can be omitted since it's being overridden on the command line (in the example) with the --volume $HOME/fah:/fah statement.

Meanwhile I have also kind of solved the whole problem as a whole by using an init container inside Kubernetes that straightens everything out privileged before the unprivileged container is started.
I'll create a PR for the README.md with a reference in due time when it's of substance. :-)

@beberg
Copy link
Contributor

beberg commented Apr 23, 2020

You are correct the UID the user chooses to run the container as, has to have read-write permissions to the persistent storage directory. The init container is a good solution in your specific case, but may not work in others. E.g. the permissions can be set during setup while creating one config.xml for each container.

How that is done varies a lot by storage system, but I should have called it out explicitly in the README. I'll fix it in the morning.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants