Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not use tools param in stream request! #2810

Open
Snowdar opened this issue Jan 9, 2025 · 0 comments
Open

Do not use tools param in stream request! #2810

Snowdar opened this issue Jan 9, 2025 · 0 comments

Comments

@Snowdar
Copy link

Snowdar commented Jan 9, 2025

request.stream = False

If you give a tools param in your request and set stream=True, then the output format will be changed by the server and you will get nothing by for grammar (no error will be raised), because the two processing are complete different in the client:

stream -> received with generator of chunks: generater -> async for chunk in result:
non-stream-> received with a fixed result chunk -> use it direct

So, I think if the server does not support stream with tools, then it will be better to return a http error than changing the return method so that the developers can know what should be done or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant