Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/add vllm deploy #82

Merged
merged 23 commits into from
Nov 6, 2023
Merged

Feature/add vllm deploy #82

merged 23 commits into from
Nov 6, 2023

Conversation

ruiyiw
Copy link
Collaborator

@ruiyiw ruiyiw commented Oct 30, 2023

Closes #77

📑 Description

✅ Checks

  • My pull request adheres to the code style of this project
  • My code requires changes to the documentation
  • I have updated the documentation as required
  • All the tests have passed
  • Branch name follows type/descript (e.g. feature/add-llm-agents)
  • Ready for code review

ℹ Additional Information

lwaekfjlk and others added 23 commits October 11, 2023 02:17
@lwaekfjlk
support qlora
26a64e5
@lwaekfjlk
upload dummy conversation data
25bda51
@lwaekfjlk
delete doc and docker
32f35c2
@lwaekfjlk
update pyproject pip install package
008100e
@lwaekfjlk
continue cleaning
57e09c6
@lwaekfjlk
delete more files
71eeb82
@lwaekfjlk
delete a format
117128f
@lwaekfjlk
add llm_deploy
b3eb7b9
@lwaekfjlk
add testing scripts
13e2973
@lwaekfjlk
update deployment readme
e74716d
@lwaekfjlk
update readme and fix some bug
e328128
@lwaekfjlk
finalize the inference and deployment based on vllm
470b034
@ruiyiw
Add babel deployment tutorial md
89330f9
@ruiyiw
add minor suggestions
f5d22c4
@lwaekfjlk
Merge branch 'main' into feature/add-vllm-deploy
16328e7
@ruiyiw
delete qlora_train.sh
d76f04f
@ruiyiw
Delete duplicate data file
e8d655b
@ruiyiw
Add tutorial for ssh tunnel
f0da0a8
@lwaekfjlk
Merge branch 'main' into feature/add-vllm-deploy
b6c03cf
@ruiyiw
Add fastchat api server tutorial
8e92192
@ruiyiw
Add fastchat api server tutorial
98d4189
@ruiyiw
Minor modification on the deployment tutorial
7dcbbe9
@ruiyiw
Merge branch 'main' into feature/add-vllm-deploy
46373f1
@ruiyiw ruiyiw merged commit aea8ad6 into main Nov 6, 2023
3 checks passed
lwaekfjlk added a commit that referenced this pull request Nov 16, 2023
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
6a6c26a 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
lwaekfjlk pushed a commit that referenced this pull request Nov 17, 2023
@Michaelvll
fix train yaml (#82)
20dcafa
lwaekfjlk added a commit that referenced this pull request Nov 17, 2023
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
9020bce 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
5a7d8f2 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
926006b 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
e6e55f6 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
4042257 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
3a9b62d 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
Signed-off-by: Haofei Yu <[email protected]>
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
86529df 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
67f790a 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
14174c8 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
33546db 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
e8581e4 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
12a6629 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
497aba1 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
c04292c 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
Signed-off-by: Haofei Yu <[email protected]>
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
f2e2133 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
(cherry picked from commit 1730759)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@Michaelvll
fix train yaml (#82)
66ed8d7
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/add vllm deploy (#82)
8d8b63f 
* support qlora

* upload dummy conversation data

* delete doc and docker

* update pyproject pip install package

* continue cleaning

* delete more files

* delete a format

* add llm_deploy

* add testing scripts

* update deployment readme

* update readme and fix some bug

* finalize the inference and deployment based on vllm

* Add babel deployment tutorial md

* add minor suggestions

* delete qlora_train.sh

* Delete duplicate data file

* Add tutorial for ssh tunnel

* Add fastchat api server tutorial

* Minor modification on the deployment tutorial

---------

Co-authored-by: lwaekfjlk <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEAT]: Deploy fine-tuned models on babel server and provide API
2 participants
@ruiyiw @lwaekfjlk