PEFT Finetune Container

[NiteeshReddyB]

How to pass a custom jsonL for finetuning job like by just uploading the jsonl file to GCS or upload the Jsonl file on huggingface and also in the args how to pass evaluation args

[alvarobartt]

Hi here @NiteeshReddyB, sorry I didn't quite get it, could you please add more details?

Do you mean reading a JSONL dataset to be used for fine-tuning? If that's the case, it would depend on which library are you using e.g. TRL's CLI at the moment only supports datasets being loaded from the Hugging Face Hub which is what we showcase in this example.

Alternatively, for the kwargs the same, depends on the library you use for training within the Hugging Face DLC; e.g. for TRL CLI you can provide a YAML file with the fine-tuning configuration as per TRL's documentation.

Anyway, if you could provide more details that would be great to be able to help you further! Thanks in advance 🤗

[NiteeshReddyB]

Yes, for a JSONL dataset to be used for fine-tuning. Got it, thanks.

[alvarobartt]

FYI I've created a dedicated example for this issue for you to check; but long-story-short you can use a custom fine-tuning script based on TRL and modify the loading function for the datasets, allowing you to load the JSONL file from a GCS bucket via load_dataset("json", data_files=[...]).

https://gist.github.com/alvarobartt/b2a3067d69622a0ef05aee3a113ce73b

Hope that helps! 🤗

Note

The example provided above is a minimal and fast reproduction of the issue to showcase that it can be done, while that may not be the best way, since we're reusing the dataset_name argument only; but ideally one should create a custom script suited to its needs. I think that being able to easily run custom code on top of the containers is really useful, so I'll be creating a new dedicated example in the repository!

[alvarobartt]

P.S. More information on the from_local_script method on python-aiplatform.

[NiteeshReddyB]

Thank you for sharing the links; they were very helpful. Does Hugging Face offer prebuilt containers for CI/CD on any cloud platforms like Azure or AWS that can be used to deploy or fine-tune models? Additionally, are there prebuilt containers available for deploying multi-models from Hugging Face?

[alvarobartt]

Anytime! 🤗

Does Hugging Face offer prebuilt containers for CI/CD on any cloud platforms like Azure or AWS that can be used to deploy or fine-tune models?

Yes, any of the Hugging Face DLCs can be used for any workload, including CI/CD; here you can find the useful links for each Cloud Provider with available DLCs:

• AWS:
  • Inference: https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-inference-containers
  • Training: https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-inference-containers
  • TGI https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-inference-containers

More information at https://github.com/aws/deep-learning-containers, https://aws.amazon.com/ai/hugging-face/, and https://huggingface.co/docs/sagemaker/en/index

• Google Cloud:
  • Inference https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/pytorch/inference
  • Training https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/pytorch/training
  • TGI https://github.com/huggingface/Google-Cloud-Containers/blob/main/containers/tgi/README.md
  • TEI https://github.com/huggingface/Google-Cloud-Containers/tree/main/containers/tei

More information at https://huggingface.co/docs/google-cloud/index, and https://cloud.google.com/deep-learning-containers/docs/choosing-container#hugging-face

Alternatively, we also have integrations with other providers as Azure.

Additionally, are there prebuilt containers available for deploying multi-models from Hugging Face?

With multi-models do you mean multi-modal? If that's the case you can do so with the TGI DLCs e.g. the Hugging Face DLC for TGI on Google Cloud us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu121.2-2.ubuntu2204.py310; as those come with VLM support for certain architectures, e.g. IDEFICS.

[NiteeshReddyB]

Thanks for sharing that really helps. Any useful links for Azure please share you have any.