Skip to content

Minimal nix-R container for efficient Data Science DevOps

Notifications You must be signed in to change notification settings

rstats-on-nix/r-minimal-alpine

Repository files navigation

🎯 Goal

This repo provisions a minimal nix-R container setup intended to run efficient data science DevOps workflows.

  • provision containers as remote builders for a nix store.
  • provision containers for CI/CD environments, e.g. for data engineering purposes.

This is an experimental proof of concept to deliver a minimal alpine image provinding nix in a multiuser docker environment.

πŸ₯— Recipe

  • Build the image on macOS or linux using pinned nix shebang scripts
# nix-shebang script
./build_container.sh alpine-nix .
  • Run the image
./run_container.sh alpine-nix

🌌 Background story

I've been reading a lot on the internet and I was surprised to find that what I wanted seemed to be a a patchwork of existing problems and solutions. Point out to me if you think the ideas are stubborn or have security holes.

  • Containers and Nix are a dream combo.
  • I wanted the official nix bash install script to install CppNix. First, because the DetSys/Lix rust installer exits and does not like multiuser docker installation out of the box.
  • Alpine images are usually intended to provide instances

πŸ› οΈ Implementation

  • Create a container from a nix-shebang shell script that boostraps the docker build environment from a reproducibly pinned default.nix, proving podman and qemu (linux) or vfkit (macOS/darwin) virtualization.

πŸ”€ Ways forward, backporting

Containers for runtimes such as Docker can also be built directly with Nix.

  • pkgs.dockerTools.buildImage can make potentially smaller docker images. So yeah, the same Dockerfile can maybe just be injected into podman run using result. Say "hi" to declarative docker.nix files for cross-platform nix builds.

  • For example, buildLayeredImage can be used to get caching based on layer content.

πŸ“š Varia, inspiration, links

Docker and Nix

Virtualization, containers and networking

Various Nix practices in software engineering

Pinning mechanisms in Nix

-> npins is successor: https://vaibhavsagar.com/blog/2018/05/27/quick-easy-nixpkgs-pinning/ https://jade.fyi/blog/pinning-packages-in-nix/

https://nixery.dev/

-> no nested virtualization

Running applications or emulators in a nested VM
Testing software releases on VMs
Reducing deployment times for training environments
Using Hyper-V isolation for containers

About

Minimal nix-R container for efficient Data Science DevOps

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published