Allow specifying model in env file

[mmurad2]

Issue description
New users don't know which model the bee-stack is using. Ideally they should have the flexibility to specify which model to use as well as modify parameters.

Ideal solution

• Include model param in env file
• Document which model(s) and settings the bee-stack is optimized for and include a warning that switching models may deteriorate experience.

Related discussion: https://github.com/i-am-bee/internal/issues/2

[jezekra1]

The model is configurable through API :)

You can create bee a with a different model using the python SDK, just switch the model (depends on your LLM_BACKEND):

https://github.com/i-am-bee/bee-python-sdk/blob/main/examples/basic_usage.py#L18

Or use this curl (for bee-stack):

# env for stack
BEE_API=localhost:4000
BEE_API_KEY=sk-proj-testkey

curl -X POST \
  "${BEE_API}/v1/assistants" \
  -H "Authorization: Bearer ${BEE_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/llama-3-1-70b-instruct",
    "tools": [
      {
        "type": "code_interpreter"
      }
    ]
  }'

You can then edit your assistant in the UI.

IMO it would make sense to make this configuratble in the UI

[mmurad2]

@jezekra1 In the short-term, can we add this as a guide to the bee-stack readme? Running the stack with a different model is a common request. Ideally, there should be a quick way to toggle between the models we optimized for at the framework level (llama and granite) while setting up the stack. There should be also a guide for advanced users to run the stack against models we didn't optimize against.

[jezekra1]

In the short-term, can we add this as a guide to the bee-stack readme?

I added the curl command to the advanced section, but I don't want to include the list of available model, as it changes often in the framework and it would get out of sync quickly.

It depends on the presets available in each adapter, for example for watsonx:
https://github.com/i-am-bee/bee-agent-framework/blob/main/src/adapters/watsonx/chatPreset.ts

Should I add this disclaimer to the README?

Ideally, there should be a quick way to toggle between the models we optimized for at the framework level (llama and granite) while setting up the stack.

We can add this setting to API and then to the startup script, here is api issue: i-am-bee/bee-api#76

[mmurad2]

I don't want to include the list of available model, as it changes often in the framework and it would get out of sync quickly

I am wondering if there is a creative solution here. At the very min, we should link to a section in the bee-framework where we keep an up-to-date list of models we are optimized against (cc: @Tomas2D ). Could we maybe have an agent that listens to changes to that section and updates the bee-stack readme as well?

[mmurad2]

Discussion: should model switching be done directly in the ui - as opposed to while configuring the bee-stack?