Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vector DB: local model (privateGPT) #8

Closed
slobentanzer opened this issue Jul 19, 2023 · 3 comments
Closed

Vector DB: local model (privateGPT) #8

slobentanzer opened this issue Jul 19, 2023 · 3 comments
Milestone
Improve vector da...

Comments

@slobentanzer
Copy link
Contributor

slobentanzer commented Jul 19, 2023 •
edited
Loading

https://github.com/imartinez/privateGPT is an example for local hosting of models for increased privacy. Could also be less expensive than relying on commercial services for the embeddings of large datasets.

LLaMA2 is open source now, could be a good alternative. Has an API on replicate: https://replicate.com/a16z-infra/llama13b-v2-chat
@slobentanzer slobentanzer converted this from a draft issue Jul 19, 2023
@slobentanzer
Copy link
Contributor Author

Of note, the replicate API is not the same as the naive model. It seems to have some additional layer that changes responses (sensitive content etc.).

@slobentanzer
Copy link
Contributor Author

Building a solution based on Xorbits Inference and Docker

@slobentanzer slobentanzer moved this to In Progress in BioCypher Development Nov 9, 2023
@slobentanzer
Copy link
Contributor Author

Using open-source embedding models is now possible via PR #24

@slobentanzer slobentanzer moved this from In Progress to Done in BioCypher Development Nov 23, 2023
slobentanzer pushed a commit that referenced this issue Dec 7, 2023
@fengsh27
Merge pull request #8 from fengsh27/wip/fengsh/mock-test-llm
5c01b32 
Mock dependencies on OpenAI and Milvus server in llm_connect test, prompts tests and benchmark tests
slobentanzer pushed a commit that referenced this issue Jun 13, 2024
@ytehran
Merge pull request #8 from mehizli/develop
ad7a0da 
Aktueller Stand
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
BioCypher Development
Status: Done
ChatGSE and biochatter - WARNING, DEPRECATED
Status: Done
Milestone
Improve vector database handling
Development

No branches or pull requests
1 participant
@slobentanzer