You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If you give a tools param in your request and set stream=True, then the output format will be changed by the server and you will get nothing by for grammar (no error will be raised), because the two processing are complete different in the client:
stream -> received with generator of chunks: generater -> async for chunk in result:
non-stream-> received with a fixed result chunk -> use it direct
So, I think if the server does not support stream with tools, then it will be better to return a http error than changing the return method so that the developers can know what should be done or not.
The text was updated successfully, but these errors were encountered:
sglang/python/sglang/srt/openai_api/adapter.py
Line 882 in b5fb4ef
If you give a tools param in your request and set stream=True, then the output format will be changed by the server and you will get nothing by
for
grammar (no error will be raised), because the two processing are complete different in the client:So, I think if the server does not support stream with tools, then it will be better to return a http error than changing the return method so that the developers can know what should be done or not.
The text was updated successfully, but these errors were encountered: