Skip to content

Commit

Permalink
docs: readme
Browse files Browse the repository at this point in the history
  • Loading branch information
Angular2Guy committed May 20, 2024
1 parent 230e127 commit 6d05a48
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This is a project to show howto use SpringAI to chat with the documents in a lib
The project uses Spring Boot with Spring AI to access OpenAI and the vector database. The Postgresql DB is used with the vector, hstore and the uuid-ossp extensions. Liquibase is used to manage the database migrations. Jpa is used for database access in the services by the rest controllers. For the frontend Angular, Angular Material and Typescript is used to create the table of documents, the upload dialog and the document chat. Angular Cli is used for the frontend build and Gradle for the project build.

### Ollama support
The project uses Spring AI to answer questions to an image database. Spring AIs Ollama support is used to run the LLava model that creates descriptions of images. The descriptions are turned in Embeddings and are stored in a vector database(PGVector in this project). The image is also stored in a relational table. The questions for an image can be entered in the frontend the backend turns the question in embeddings and the vector database is searched for the nearest neighbors. The best matches are returned as descriptions and images to be displayed in the frontend.
Ollama is used to use locally run AI/LLM models. Ollama can be installed or run as a Docker image in a local machine. Ollama supports a library of AI/LLM models for different use cases. Models like LLava for image description, falcon for RAG based question answering, sqlcoder for Sql generation and mixtral for function calling can be used. Ollama can use GPUs if they are available and works on CPU without them. Spring AI has Ollama support that make the use similar to using an AI service. On current CPUs there are often performance issues. The CPU providers want to add AI engines to their CPUs in the future to solve these issues.

### Image library search
The project uses Spring Boot with Spring AI to question a image database that uses Postgresql with the PGVector extension and Ollama. The for the imported and resized images are descriptions generated with the LLava model that runs locally on Ollama. These descriptons are turned in Embeddings and are stored with the description in the vector database. The image and metadata is stored in the relational database. A question to the image database is turned in Embeddings and the vector database is queried for the nearest neighbor. The best matches are returned with image and description. The result is displayed in the frontend. This enables are new kind of image search that was not possible before AI/LLMs became usable.
Expand Down

0 comments on commit 6d05a48

Please sign in to comment.