Feature/llama factory llama2 pipeline #89

Jasonqi146 · 2023-11-07T06:16:46Z

Closes #88

📑 Description

Added training script for training llama-2-13b.
Added and registered dummy dataset in /data directory.
Added and registered training dataset in /data directory.
Modified original sft to incorporate wandb.
Added global rank checking for determining logger.

✅ Checks

My pull request adheres to the code style of this project
My code requires changes to the documentation
I have updated the documentation as required
All the tests have passed
Branch name follows type/descript (e.g. feature/add-llm-agents)
Ready for code review

ℹ Additional Information

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k (cherry picked from commit c769638)

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k (cherry picked from commit c769638) Signed-off-by: Haofei Yu <[email protected]>

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k (cherry picked from commit c769638)

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k (cherry picked from commit c769638) Signed-off-by: Haofei Yu <[email protected]>

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k (cherry picked from commit c769638)

* added llama-factory under llm_rl * added sft training bash * added datasets from llama-factory; will delete later * finished llama-2-13b train and inference * fixed minor errors * changed config * added deepspeed config * added more training config to train bash * adding fix for wandb tags and distributed ranks * added fastchat data to replicate training for 2k

Jasonqi146 and others added 12 commits November 6, 2023 20:05

added llama-factory under llm_rl

80a3b65

added sft training bash

5ca81da

added datasets from llama-factory; will delete later

6e65907

finished llama-2-13b train and inference

a984527

fixed minor errors

9db6bbe

changed config

2441021

added deepspeed config

847abf4

Merge branch 'main' into feature/llama-factory-llama2-pipeline

083e0f4

added more training config to train bash

cd4e0c2

adding fix for wandb tags and distributed ranks

7b73354

added fastchat data to replicate training for 2k

cc1f259

merged with previous changes

1aac286

Jasonqi146 merged commit 6a603ca into main Nov 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/llama factory llama2 pipeline #89

Feature/llama factory llama2 pipeline #89

Jasonqi146 commented Nov 7, 2023 •

edited

Loading

Feature/llama factory llama2 pipeline #89

Feature/llama factory llama2 pipeline #89

Conversation

Jasonqi146 commented Nov 7, 2023 • edited Loading

📑 Description

✅ Checks

ℹ Additional Information

Jasonqi146 commented Nov 7, 2023 •

edited

Loading