Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitHub action to auto-build Docker images #49

Merged
merged 15 commits into from
Nov 7, 2023
51 changes: 51 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
konstantinjdobler marked this conversation as resolved.
Show resolved Hide resolved
name: Build and push Docker image
on:
push:
branches:
- main
- add/docker_ci
paths:
- Dockerfile
- conda-lock.yml
jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Maximize build space
uses: easimon/maximize-build-space@master
with:
remove-dotnet: "true"
remove-codeql: "true"
remove-haskell: "true"
remove-android: "true"
overprovision-lvm: "true"
remove-docker-images: "true"
- name: Checkout
uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: amd64
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
registry: ${{secrets.DOCKER_REGISTRY}}
username: ${{secrets.DOCKER_REGISTRY_USERNAME}}
password: ${{secrets.DOCKER_REGISTRY_TOKEN}}
- name: Get torch version for amd
run: "echo TORCH_VERSION_AMD=$(cat conda-lock.yml | grep -E -A 4 'name: pytorch$' |
tail -n1 | grep -Eo [0-9]+.[0-9]+.[0-9]+) >> $GITHUB_ENV"
- name: Get cuda version for amd
run: "echo CUDA_VERSION_AMD=$(cat conda-lock.yml | grep -E -A 4 'name:
pytorch-cuda$' | tail -n1 | grep -Eo [0-9]+.[0-9]+) >> $GITHUB_ENV"
- name: Build and push AMD
uses: docker/build-push-action@v4
with:
context: .
platforms: linux/amd64
push: true
tags: konstantinjdobler/nlp-research-template:pytorch${{env.TORCH_VERSION_AMD}}-cuda${{env.CUDA_VERSION_AMD}}, konstantinjdobler/nlp-research-template:latest

27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,3 +232,30 @@ Sometimes it's just quicker or unavoidable to create an environment via `conda-l
### Code style

We use the `ruff` linter and `black` formatter. You should install their VS Code extensions and enable "Format on Save" inside VS Code.


## Continuous Integration and Deployment

Our project uses GitHub Actions for CI/CD to automate the building and pushing of our Docker images to Docker Hub. This ensures that our Docker images are always up-to-date with the latest dependencies specified in `conda-lock.yml`.

### Prerequisites for CI/CD

To work with this CI/CD setup, you need to:

- Set the following secrets in your GitHub repository:
- `DOCKER_REGISTRY`: The Docker registry URL (if using Docker Hub, this is not needed).
- `DOCKER_REGISTRY_USERNAME`: Your Docker Hub username.
- `DOCKER_REGISTRY_TOKEN`: Your Docker Hub access token or password.

### How to Update Docker Images

To update the Docker image:

1. Make necessary changes to the `Dockerfile` or update dependencies in the `environment.yml`.
2. Generate a new `conda-lock.yml` by running `conda-lock -f environment.yml`.
3. Commit and push the changes to the `main` branch.
4. The GitHub Actions workflow will automatically build and push the new Docker image to Docker Hub.

### Docker Tags

The Docker images are tagged with the PyTorch and CUDA versions extracted from `conda-lock.yml`, as well as a `latest` tag for the most recent build. Use the specific tags if you need a particular version of PyTorch or CUDA, or use the `latest` tag for the most recent build.