Skip to content

Building Apptainer Images

Jump to bottom
Christian Muise edited this page Oct 12, 2022 · 2 revisions

Planner Apptainer Images

Many planners are available as Apptainer (formerly Singularity) images. If you are familiar with Docker, Apptainer is a similar type of container software that is geared more towards high-performance computing. Apptainer has low overhead which makes it suitable for experimental evaluations.

Installing Apptainer

Apptainer offers prepacked versions as *.deb and *.rpm files on their release page. For example, to install Apptainer on a Debian-based system:

wget https://github.com/apptainer/apptainer/releases/download/v1.1.0/apptainer_1.1.0_amd64.deb
dpkg -i apptainer_1.1.0_amd64.deb

Apptainer is developed to run on a Linux host system but there are experimental ways to run it on macOS and Windows. There is also the option of running Apptainer inside a virtual machine on such systems but this is not recommended for timing-critical experiments.

After the installation process, you can test your installation by downloading a minimal Linux container and printing its version:

    apptainer exec library://alpine cat /etc/alpine-release

Obtaining a Apptainer Image

To run a planner with Apptainer, you need its image file. For some planners you can get them precompiled, for others you need to build them from a recipe file. However you obtain the image, it will create a file on your hard disk that is a self-contained version of the planner, including everything needed to run it.

A) Images from Singularity Hub

Singularity Hub is a service that builds and hosts Singularity images. You can download a planner image like this:

    apptainer pull --name downward.simg shub://aibasel/downward

B) Images from Recipe Files

To build an Apptainer image yourself, you need access to a recipe file. For example, Apptainer recipes for the participants of the IPC 2018 are available from the IPC 2018 website.

You can then use Apptainer to build the planner. Be aware that this step requires root access. If you do not trust the source of the code and everyone able to modify it, you should run this call isolated in a virtual machine (example VM from the IPC).

    wget https://bitbucket.org/ipc2018-classical/team47/raw/ipc-2018-seq-agl/Singularity
    sudo apptainer build lapkt-bfws-preference.simg Singularity

Running an Apptainer Image

Once downloaded, the planner images acts like a binary on your system, i.e., you can call them like any other binary and they will run with the access rights of the user that called the process. They will also spawn processes in the same way that running the same planner natively would.

WARNING: Apptainer only has access to /tmp and /home directories by default. If you are using arguments to your built image that point to files outside of those locations, the built image will not have access to those files.

The necessary command-line parameters depend on the planner. For example, Fast Downward is a planning system, not a single planner, so it requires parameters to select a search method, heuristic, etc. However, all IPC planners use a common interface and can be called like this (except in the cost-bounded track where they take a fourth argument):

    ./planner.simg path/to/domain.pddl path/to/problem.pddl path/where/to/write/plan

You can explore the shell of the build image by running the following:

    apptainer shell planner.simg

Note that some planners require additional isolation or expect to run in a time limited setting. Refer to the next section for more advanced ways of running Apptainer images.

Advanced Options when Running a Planner

Some planners expect there to be a time or memory limit, others may create temporary files that may become big. Some planner also require either absolute or relative paths in the argument that passes in the PDDL files, and some expect the PDDL file to be in the current working directory. The following call creates two directories, a working directory rundir and a temporary directory tmpdir, then limits each process to a time 30 minutes and a memory usage of 8 GiB. Note that these limits are per process, so a planner that runs multiple processes (like all portfolio planners) can use 30 minutes in each process. More detailed resource management requires process monitoring with cgroups.

    mkdir rundir
    mkdir tmpdir
    cp path/to/domain.pddl rundir
    cp path/to/problem.pddl rundir
    ABS_RUNDIR=$(realpath rundir)
    ABS_TMPDIR=$(realpath tmpdir)
    DOMAIN="$ABS_RUNDIR/domain.pddl"
    PROBLEM="$ABS_RUNDIR/problem.pddl"
    PLANFILE="$ABS_RUNDIR/sas_plan"
    ulimit -t 1800     # limit runtime to 30 minutes per process
    ulimit -v 8388608  # limit memory usage to 8 GiB per process
    ulimit -f 409600   # limit size of created files to 400 MiB per file
    ulimit -c 0        # switch off core dumps
    singularity run -C -B $ABS_TMPDIR:/tmp -H $ABS_WORK_DIR planner.simg $DOMAIN $PROBLEM $PLANFILE

Writing your own Container Files

You can create your own Apptainer recipes by following the example of the IPC 2018 planner, for example the demo submission. Apptainer images can be bootstraped from docker images, copying all files in the docker image into the container. Afterwards, the two most relevant sections in a recipe file are %post and %runscript. The former is executed inside the container when the container is built and is responsible for installing all dependencies and compiling the planner. The latter is called whenever the image is run and is responsible for calling the actual planner with the command-line arguments provided by the user.

For more detailed instructions, see the Singularity documentation.

Creating an Apptainer image from a Docker image

Sometimes it is easier to create a Docker image for your planner. We still recommend using an Apptainer image for running experiments for its better behavior in high-performance settings. It is easy to create an Apptainer image from an existing docker image and the resulting image has the performance characteristics of an Apptainer image (going through a Docker image has no negative effect).

To build an Apptainer image from a Docker image called planner1:latest on Dockerhub, use

apptainer build planner1.sif docker://planner1:latest

If the Docker image is not on Dockerhub but only in your local Docker registry (i.e., if you built and tagged the Docker locally), use

apptainer build planner1.sif docker-daemon://planner1:latest

Keeping the Image Files Small

When creating a planner image, keep in mind that the users of that image might have to move that file around within their compute cluster, or may want to compare a large number of planners. For these reasons, try to keep your planner image as small as possible. The following tricks can help with this (both for Docker and Apptainer):

  1. Delete unnecessary files. For example, do not keep a copy of the repository or compiled object files around. You can delete all files other than the ones required to run the planner.

  2. Uninstall unnecessary software. For example, after compilation your planner probably does not need g++ anymore. For Docker images, it can help to pack installing the software, using it, and uninstalling it into a single step as every step increases the image size (the final size increases even if you delete a file in a later step). Apptainer does not use layers such as Docker, so only the files that remain in the image count towards its size.

  3. Use multi-stage builds. Multi-stage builds (Apptainer, Docker) combine the two suggestions above. When the build is done in multiple stages, the first stage can compile the planner and the second stage only copies the relevant files from the first stage. The first stage is then discarded, so no clean-up is necessary.

  4. Use small packages. For example, install python-minimal instead of python.

  5. Use a small base image. For example, consider using Alpine Linux (2.7 MiB) instead of Ubuntu (26.8 MiB) as a base. However, be aware that Alpine Linux uses musl instead of glibc.

  6. Strip your binaries. Compiled planners often contain debug symbols and other information that is not required to run the planner. Using strip you can get rid of this without affecting the behavior of your binary.

Clone this wiki locally