SFT Training update tutorials #769
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tengomucho
force-pushed
the
training-update-tutorials
branch
from
81dd858 to
64a9623
Compare
dacorvo
requested changes
Jan 28, 2025
setup.py
Outdated
| @@ -19,6 +19,8 @@ | |||
| "huggingface_hub >= 0.20.1", | |||
| "numpy>=1.22.2, <=1.25.2", | |||
| "protobuf>=3.20.3, <4", | |||
| "trl == 0.11.4", | |||
There was a problem hiding this comment.
These are the global, bare minimum dependencies: we should not add optional components here.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
dacorvo
approved these changes
Jan 28, 2025
Comment on lines
+59
to
+62
| 5. Make sure you have the `training` extra installed, to get all the necessary dependencies: | ||
| ```bash | ||
| python -m pip install .[training] | ||
| ``` |
| @@ -63,7 +73,7 @@ Example: | |||
| "context": "", | |||
| "response": ( | |||
| "World of warcraft is a massive online multi player role playing game. " | |||
| "It was released in 2004 by blizarre entertainment" | |||
| "It was released in 2004 by bizarre entertainment" | |||
There was a problem hiding this comment.
Suggested change
| "It was released in 2004 by bizarre entertainment" | |
| "It was released in 2004 by Blizzard Entertainment" |
There was a problem hiding this comment.
Nope, that is what is actually in the Dolly dataset! See here 🤷
There was a problem hiding this comment.
As a former big player of WoW I feel attacked.
| --gradient_accumulation_steps $GRADIENT_ACCUMULATION_STEPS \ | ||
| --gradient_checkpointing true \ | ||
| --bf16 \ | ||
| --zero_1 false \ |
| @@ -50,9 +50,15 @@ | |||
| "hf_doc_builder @ git+https://github.com/huggingface/doc-builder.git", | |||
| ] | |||
|
|
|||
| TRAINING_REQUIRES = [ | |||
| "trl == 0.11.4", | |||
| "peft == 0.14.0", | |||
There was a problem hiding this comment.
Can we add neuronx_distributed as well?
Slurm can be used for multi-node training, but not relevant in the tutorial, so it is removed.
The shell script is to launch precompilation and fine-tuning.
- reword inference hw suggestion - adapt code to use the merged model directory
Peft was already in the AMI, but hte version was not fixed, and TRL was not installed, but it is required for the SFT LoRA fine tune LLM tutorial.
Instead of adding peft and trl as package dependencies, these are isolated in the training extra.
tengomucho
force-pushed
the
training-update-tutorials
branch
from
544da8d to
15408fe
Compare
michaelbenayoun
approved these changes
Jan 29, 2025
| @@ -63,7 +73,7 @@ Example: | |||
| "context": "", | |||
| "response": ( | |||
| "World of warcraft is a massive online multi player role playing game. " | |||
| "It was released in 2004 by blizarre entertainment" | |||
| "It was released in 2004 by bizarre entertainment" | |||
There was a problem hiding this comment.
As a former big player of WoW I feel attacked.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR revisits the SFT Training tutorial of Llama3-8B. Few highlights of the changes:
Note that there is still an issue with the training: the loss does NOT decrease during fine-tune (cc @michaelbenayoun).
Before submitting