Skip to content

Latest commit

 

History

History
67 lines (40 loc) · 2.22 KB

README.md

File metadata and controls

67 lines (40 loc) · 2.22 KB

T5 Fine-Tuning with QLoRA

Overview

This repository contains scripts and resources for fine-tuning the Google T5 Mini model. Due to the limitations of the NVIDIA RTX 2070 GPU, QLoRA (Quantized Low-Rank Adaptation) is used to make the fine-tuning process more memory-efficient while maintaining performance.

Motivation

The RTX 2070 GPU, with its 8 GB memory, poses challenges for traditional fine-tuning methods, especially with models like T5. QLoRA offers a solution by reducing the memory requirements, enabling effective fine-tuning on limited hardware.

Structure

  • EDA.ipynb: Notebook for exploratory data analysis, providing insights into the dataset before fine-tuning.

  • benchmark.py: Script to benchmark the fine-tuned model and evaluate performance improvements.

  • dataimport.py: Handles data import and preprocessing, preparing it for training.

  • fine-tune.py: Script for fine-tuning the T5 Mini model using QLoRA, optimized for the RTX 2070 GPU.

Fine-Tuning Approach

Why QLoRA?

Given the memory limitations of the RTX 2070, QLoRA is employed to reduce the memory footprint of the fine-tuning process without compromising on model accuracy.

Steps

  1. Model Selection: Starting with the T5 Mini model for its balance of performance and efficiency.
  2. Data Preparation: Data is cleaned and preprocessed with dataimport.py.
  3. Fine-Tuning: The model is fine-tuned using fine-tune.py, utilizing QLoRA for efficiency.
  4. Benchmarking: Performance is evaluated with benchmark.py.

Requirements

  • Python 3.8+
  • PyTorch
  • Hugging Face Transformers
  • NVIDIA RTX 2070 GPU or similar

Installation

Clone the repository and install the required dependencies:

git clone https://github.com/yourusername/your-repo-name.git
cd your-repo-name
pip install -r requirements.txt

Usage

  1. Prepare data with dataimport.py.
  2. Fine-tune the model using fine-tune.py.
  3. Benchmark the results with benchmark.py.

Results

Performance metrics and results from the fine-tuning process will be documented here.

License

This project is licensed under the MIT License.

This version is more direct and less descriptive, focusing on key information and instructions.