Completed Modules: 1,2,3

[avisha-datascientist]

December 2024 Student Submission

Module Completed

• Module 1: Instruction Tuning
• Module 2: Preference Alignment
• Module 3: Parameter-efficient Fine-tuning
• Module 4: Evaluation
• Module 5: Vision-language Models
• Module 6: Synthetic Datasets
• Module 7: Inference
• Module 8: Deployment

Changes Made

Describe what you've done in this PR:

1. What concepts did you learn? - instruction tuning, ORPO, DPO, LoRA, QLoRA
2. What changes or additions did you make? - tried on example notebooks on different datasets and use cases
3. Any challenges you faced?

Notebooks Added/Modified

List any notebooks you've added or modified:

• Added new example in 1_instruction_tuning/student_examples/avisha-datascientist/sft_finetuning_datascienceqa.ipynb, 2_preference_alignment/student_examples/avisha-datascientist/smolllm2-orpo-truthydataset.ipynb
• Modified existing notebook with additional examples 3_parameter_efficient_finetuning/student_examples/avisha-datascientist/finetune_sft_peft.ipynb
• Added documentation or comments

Checklist

• I have read the module materials
• My code runs without errors
• I have pushed models and datasets to the huggingface hub
• My PR is based on the december-2024 branch

Additional Notes

Any other information that might be helpful for reviewers:

[burtenshaw]

Amazing work @avisha-datascientist !

Would you like to take part in peer review? If so, mention me on a PR from another student, review it, and I'll get a student to review yours.

[HSILA]

Maybe #161 ?

Originally posted by @burtenshaw in #157 (comment)

Great job @avisha-datascientist
I reviewed this days ago, but I was awaiting an actual review request on this PR; sorry for the delay. Feel free to merge, @burtenshaw.