"bee" able to change the model from llama3.1 by default for ollama

[jjasghar]

I was attempting to use the granite3.0-8b-dense model, and it seems there is no way to put in a custom model. When I attempted use ollama the error came back with:

bee-api-1  | {"level":"error","time":"2024-11-20T19:46:00.849Z","hostname":"8d4ea71dff49","name":"bee-api","queueName":"runs","job":{"id":"run_673e3c77d8ebcc81d5182868","name":"runs","failedReason":"model 'llama3.1' not found","data":{"runId":"run_673e3c77d8ebcc81d5182868"}},"err":{"type":"ResponseError","message":"model 'llama3.1' not found","stack":"ResponseError: model 'llama3.1' not found\n    at checkOk (file:///app/node_modules/.pnpm/[email protected]/node_modules/ollama/dist/shared/ollama.133b951a.mjs:70:9)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async post (file:///app/node_modules/.pnpm/[email protected]/node_modules/ollama/dist/shared/ollama.133b951a.mjs:118:3)\n    at async Ollama.show (file:///app/node_modules/.pnpm/[email protected]/node_modules/ollama/dist/shared/ollama.133b951a.mjs:383:22)\n    at OllamaChatLLM.meta (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@googleapi_ia5npp4g5tjlrpr3a7futx3n24/node_modules/bee-agent-framework/src/adapters/ollama/chat.ts:148:19)\n    at TokenMemory.add (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@googleapi_ia5npp4g5tjlrpr3a7futx3n24/node_modules/bee-agent-framework/src/memory/tokenMemory.ts:85:20)\n    at TokenMemory.addMany (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@googleapi_ia5npp4g5tjlrpr3a7futx3n24/node_modules/bee-agent-framework/src/memory/base.ts:43:7)\n    at async executeRun (file:///app/dist/runs/execution/execute.js:75:3)\n    at async file:///app/dist/runs/jobs/runs.queue.js:49:7\n    at Worker.processJob (/app/node_modules/.pnpm/[email protected]/node_modules/bullmq/src/classes/worker.ts:786:22)","error":"model 'llama3.1' not found","status_code":404,"name":"ResponseError"},"msg":"Job failed"}

I have llama3 in ollama but not 3.1 which also seems weird, that it's hard coded o a specific version of llama. This opens up the issue with not being able to point to any of my models on that ollama uses.

Or am I missing something?

[mmurad2]

@Tomas2D do you have a view on this?

[Tomas2D]

It is mentioned in the README (https://github.com/i-am-bee/bee-stack?tab=readme-ov-file#custom-models)
Related issue: #36

TLDR: we would like to allow users to change the model within the UI