Skip to content

Docker apps

Jump to bottom
David Anderson edited this page Oct 8, 2024 · 14 revisions

You can run Docker apps under BOINC. To do so:

  • Develop your application in the software environment of your choice (say, particular versions of Linux and Python, with particular libraries and packages installed).

  • Package this environment as a Docker image: i.e. write a Dockerfile that builds the environment.

  • Create BOINC app versions that combine your Docker image (and other files if needed) with a "Docker wrapper" program (supplied by BOINC) that interfaces between Docker and the BOINC client, running your jobs in Docker containers.

Your application can then run on all major platforms (Linux, Windows, Mac OS). In that sense it's similar to BOINC's support for apps that run in VirtualBox virtual machines. However, the Docker approach has several advantages:

  • Docker apps can access GPUs.
  • Docker apps use much less disk space (tens of MBs rather than GBs).
  • Starting a Docker container takes less time than starting a virtual machine.

The remainder of this document describes BOINC's support for Docker apps. For a simple example, see the Docker app cookbook.

The Docker wrapper

The Docker wrapper (docker_wrapper) interfaces BOINC to Docker. It is the main program of Docker apps.

The Docker wrapper reads a config file, job.toml. This file, which is in TOML format, can contain the following items:

slot_dir_mount = "/app/slot"
project_dir_mount = "/app/project"

Mounts the job's slot or project directory at the given mount point in the container.

copy_to_container = [
    {src="in", dst="/app/in"}
    ...
]

Before the job starts, copy the given file(s) (in the slot directory) to the given path in the container.

copy_from_container = [
    {src="/app/out", dst="out"}
    ...
]

After the job has completed successfully, copy the files at the given paths in the container to the given names in the slot directory.

use_gpu = true

Allow GPU access from the container.

checkpoint_period = 3600

Specify a checkpoint period, overriding the computing preferences.

File access options

There are two to handle input and output files in Docker apps.

Copy access

In this model, files are copied between the slot directory and the Docker container.

Input files are marked as <copy_file/>, so that the slot directory contains the files themselves, rather than links.

The config file job.toml has copy_to_container commands to copy input files to the container, and copy_from_container commands to copy output files from the container to the slot directory.

In this model, input files are copied twice and output files are copied once. If files are large (GB scale) this results in

  • lots of disk activity at job start and/or end
  • high disk space usage

However, the model has the advantage that the code in your container doesn't have to know about how BOINC stores files.

Direct access

In this model, there is a single of each file, and there is no copying.

The slot and project directories must be mounted in the container. To do this job.toml should contain:

slot_dir_mount = "/app/slot"
project_dir_mount = "/app/project"

Input files should not be marked as <copy_file/>. Your container must contain a script that converts BOINC's link files to physical names:

#! /bin/sh

sed 's/<soft_link>..\/..\/projects\///; s/[^\/]*\///; s/<\/soft_link>//' $1 | tr -d '\r\n'

and the main program in the container must be a shell script that uses this to resolve filenames, e.g.

#! /bin/sh

execpath='project/'`./boinc_resolve slot/worker`
inpath='project/'`./boinc_resolve slot/in`
outpath='project/'`./boinc_resolve slot/out`

$execpath $inpath $outpath

App packaging options

A BOINC job involves

  • An app version.
  • A workunit.

Each of these is a collection of files. The files in an app version are code-signed. This is normally done manually; it prevents hackers from using your project to run malware, even if they are able to break into your server.

The files in an app version are cached on the client. They are deleted only when the app version has been superceded by a later version. Workunit files are deleted after a job is finished, unless they are marked as <sticky/> in the job's input template.

The files of a Docker app can be divided between app version and workunit in several different ways.

Standard model

In this model, the app version contains

  • Dockerfile version and possible a new app.
  • executable
  • docker_wrapper
  • job.toml

and the workunit contains

  • input files

The docker image name is boinc_<project>_<appname>_<planclass>_<version>. A single Docker image is shared among all jobs (possibly concurrent) using the app version.

Each job has its own Docker container, whose name is boinc_<resultname>.

To deploy a new application, you need to create a new app. To deploy a new version, you need to create a new app version. Both require login access to the BOINC server.

Semi-universal model

This model makes it possible to deploy new versions of a given app without login access to the BOINC server.

The app version has

  • docker_wrapper

The workunit has

  • Dockerfile
  • executable
  • job.toml
  • input files

The first three can be sticky to minimize file transfers.

In this case the image name is boinc_<wuname>. A new image will be built for each job.

The executable is not code-signed (which matters less since it's run in a container).

We call this semi-universal because there's still a separate app for each application. You can use validators and assimilators that are specific to the applications.

Universal model

This is the same as the semi-universal model except that there is a single app object. You can run unrelated applications under the same app.

This eliminates the need to create new app objects. However, it means that the validator and assimilator for the universal app must be able to handle the results of all applications.

Home

Clone this wiki locally