Skip to content

Commit

Permalink
Update README.md (#136)
Browse files Browse the repository at this point in the history
* Update `README.md` to remove unreleased TPU DLCs

* Fix formatting in `examples/cloud-run/README.md`

* Update `containers/pytorch/inference/README.md`

* Update `README.md`

* Update `README.md`

* Update `docs/source/containers/available.mdx`
  • Loading branch information
alvarobartt authored Jan 3, 2025
1 parent 951dd3b commit 4a97ad3
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 21 deletions.
18 changes: 8 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@ The [Google-Cloud-Containers](https://github.com/huggingface/Google-Cloud-Contai
- Training
- [PyTorch](./containers/pytorch/training/README.md)
- GPU
- TPU
- TPU (soon to be released)
- Inference
- [PyTorch](./containers/pytorch/inference/README.md)
- CPU
- GPU
- [Text Generation Inference](./containers/tgi/README.md)
- GPU
- TPU
- TPU (soon to be released)
- [Text Embeddings Inference](./containers/tei/README.md)
- CPU
- GPU
Expand All @@ -25,14 +25,12 @@ The [Google-Cloud-Containers](https://github.com/huggingface/Google-Cloud-Contai

| Container URI | Path | Framework | Type | Accelerator |
| --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | --------- | ----------- |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu124.2-3.ubuntu2204.py311 | [text-generation-inference-gpu.2.3.1](./containers/tgi/gpu/2.3.1/Dockerfile) | TGI | Inference | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cu122.1-4.ubuntu2204 | [text-embeddings-inference-gpu.1.4.0](./containers/tei/gpu/1.4.0/Dockerfile) | TEI | Inference | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cpu.1-4 | [text-embeddings-inference-cpu.1.4.0](./containers/tei/cpu/1.4.0/Dockerfile) | TEI | Inference | CPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu124.2-4.ubuntu2204.py311 | [text-generation-inference-gpu.2.4.0](./containers/tgi/gpu/2.4.0/Dockerfile) | TGI | Inference | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cu122.1-5.ubuntu2204 | [text-embeddings-inference-gpu.1.5.1](./containers/tei/gpu/1.5.1/Dockerfile) | TEI | Inference | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cpu.1-5 | [text-embeddings-inference-cpu.1.5.1](./containers/tei/cpu/1.5.1/Dockerfile) | TEI | Inference | CPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-training-cu121.2-3.transformers.4-42.ubuntu2204.py310 | [huggingface-pytorch-training-gpu.2.3.0.transformers.4.42.3.py310](./containers/pytorch/training/gpu/2.3.0/transformers/4.42.3/py310/Dockerfile) | PyTorch | Training | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-44.ubuntu2204.py311 | [huggingface-pytorch-inference-gpu.2.2.2.transformers.4.44.0.py311](./containers/pytorch/inference/gpu/2.2.2/transformers/4.44.0/py311/Dockerfile) | PyTorch | Inference | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-2.transformers.4-44.ubuntu2204.py311 | [huggingface-pytorch-inference-cpu.2.2.2.transformers.4.44.0.py311](./containers/pytorch/inference/cpu/2.2.2/transformers/4.44.0/py311/Dockerfile) | PyTorch | Inference | CPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-tpu.0.2.2.py310 | [huggingface-text-generation-inference-tpu.0.2.2.py310](./containers/tgi/tpu/0.2.2/Dockerfile) | TGI | Inference | TPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-training-tpu.2.5.1.transformers.4.46.3.py310 | [huggingface-pytorch-training-tpu.2.5.1.transformers.4.46.3.py310](./containers/tgi/tpu/0.2.2/Dockerfile) | PyTorch | Training | TPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-3.transformers.4-46.ubuntu2204.py311 | [huggingface-pytorch-inference-gpu.2.3.1.transformers.4.46.1.py311](./containers/pytorch/inference/gpu/2.3.1/transformers/4.46.1/py311/Dockerfile) | PyTorch | Inference | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-3.transformers.4-46.ubuntu2204.py311 | [huggingface-pytorch-inference-cpu.2.3.1.transformers.4.46.1.py311](./containers/pytorch/inference/cpu/2.3.1/transformers/4.46.1/py311/Dockerfile) | PyTorch | Inference | CPU |

> [!NOTE]
> The listing above only contains the latest version of each of the Hugging Face DLCs, the full listing of the available published containers in Google Cloud can be found either in the [Deep Learning Containers Documentation](https://cloud.google.com/deep-learning-containers/docs/choosing-container#hugging-face), in the [Google Cloud Artifact Registry](https://console.cloud.google.com/artifacts/docker/deeplearning-platform-release/us/gcr.io) or via the `gcloud container images list --repository="us-docker.pkg.dev/deeplearning-platform-release/gcr.io" | grep "huggingface-"` command.
Expand Down Expand Up @@ -71,7 +69,7 @@ The [`examples`](./examples) directory contains examples for using the container
| Cloud Run | [examples/cloud-run/deploy-gemma-2-on-cloud-run](./examples/cloud-run/deploy-gemma-2-on-cloud-run) | Deploy Gemma2 9B with TGI DLC on Cloud Run |
| Cloud Run | [examples/cloud-run/deploy-llama-3-1-on-cloud-run](./examples/cloud-run/deploy-llama-3-1-on-cloud-run) | Deploy Llama 3.1 8B with TGI DLC on Cloud Run |

### Evaluation
### Evaluation Examples

| Service | Example | Title |
| --------- | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------- |
Expand Down
8 changes: 4 additions & 4 deletions containers/pytorch/inference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Before running this container, you will need to select any supported model from
-e HF_MODEL_ID=distilbert/distilbert-base-uncased-finetuned-sst-2-english \
-e HF_TASK=text-classification \
--platform linux/amd64 \
us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-2.transformers.4-44.ubuntu2204.py311
us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-3.transformers.4-46.ubuntu2204.py311
```

- **GPU**: Note that here you need to have an instance with at least one NVIDIA GPU and to set the `--gpus all` flag within the `docker run` command, as well as using the GPU-compatible container.
Expand All @@ -44,7 +44,7 @@ Before running this container, you will need to select any supported model from
-e HF_MODEL_ID=distilbert/distilbert-base-uncased-finetuned-sst-2-english \
-e HF_TASK=text-classification \
--platform linux/amd64 \
us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-44.ubuntu2204.py311
us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-3.transformers.4-46.ubuntu2204.py311
```

> [!NOTE]
Expand Down Expand Up @@ -79,11 +79,11 @@ The PyTorch Training containers come with two different containers depending on
- **CPU**

```bash
docker build -t us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-2.transformers.4-44.ubuntu2204.py311 -f containers/pytorch/inference/cpu/2.2.2/transformers/4.44.0/py311/Dockerfile --platform linux/amd64 .
docker build -t us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-3.transformers.4-46.ubuntu2204.py311 -f containers/pytorch/inference/cpu/2.3.1/transformers/4.46.1/py311/Dockerfile --platform linux/amd64 .
```

- **GPU**

```bash
docker build -t us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-44.ubuntu2204.py311 -f containers/pytorch/inference/gpu/2.2.2/transformers/4.44.0/py311/Dockerfile --platform linux/amd64 .
docker build -t us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-3.transformers.4-46.ubuntu2204.py311 -f containers/pytorch/inference/gpu/2.3.1/transformers/4.46.1/py311/Dockerfile --platform linux/amd64 .
```
10 changes: 5 additions & 5 deletions docs/source/containers/available.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,21 @@ The listing below only contains the latest version of each one of the Hugging Fa

| Container URI | Path | Accelerator |
| ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu124.2-3.ubuntu2204.py311 | [text-generation-inference-gpu.2.3.1](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/tgi/gpu/2.3.1/Dockerfile) | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu124.2-4.ubuntu2204.py311 | [text-generation-inference-gpu.2.4.0](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/tgi/gpu/2.4.0/Dockerfile) | GPU |

## Text Embeddings Inference (TEI)

| Container URI | Path | Accelerator |
| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cu122.1-4.ubuntu2204 | [text-embeddings-inference-gpu.1.4.0](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/tei/gpu/1.4.0/Dockerfile) | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cpu.1-4 | [text-embeddings-inference-cpu.1.4.0](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/tei/cpu/1.4.0/Dockerfile) | CPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cu122.1-5.ubuntu2204 | [text-embeddings-inference-gpu.1.5.1](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/tei/gpu/1.5.1/Dockerfile) | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cpu.1-5 | [text-embeddings-inference-cpu.1.5.1](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/tei/cpu/1.5.1/Dockerfile) | CPU |

## PyTorch Inference

| Container URI | Path | Accelerator |
| --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-44.ubuntu2204.py311 | [huggingface-pytorch-inference-gpu.2.2.2.transformers.4.44.0.py311](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/pytorch/inference/gpu/2.2.2/transformers/4.44.0/py311/Dockerfile) | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-2.transformers.4-44.ubuntu2204.py311 | [huggingface-pytorch-inference-cpu.2.2.2.transformers.4.44.0.py311](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/pytorch/inference/cpu/2.2.2/transformers/4.44.0/py311/Dockerfile) | CPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-3.transformers.4-46.ubuntu2204.py311 | [huggingface-pytorch-inference-gpu.2.3.1.transformers.4.46.1.py311](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/pytorch/inference/gpu/2.3.1/transformers/4.46.1/py311/Dockerfile) | GPU |
| us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-3.transformers.4-46.ubuntu2204.py311 | [huggingface-pytorch-inference-cpu.2.3.1.transformers.4.46.1.py311](https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/pytorch/inference/cpu/2.3.1/transformers/4.46.1/py311/Dockerfile) | CPU |

## PyTorch Training

Expand Down
3 changes: 1 addition & 2 deletions examples/cloud-run/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ This directory contains usage examples of the Hugging Face Deep Learning Contain

## Training Examples

Coming soon!

Coming soon!

0 comments on commit 4a97ad3

Please sign in to comment.