trunction flag is missing from /v1/chat/completions

[vitalyshalumov]

System Info

I want to be able to truncate input without raising an error using /v1/chat/completions. Is this possible?

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

input that is longer than allowed

Expected behavior

truncation from the top.

[drbh]

Hi @vitalyshalumov thanks for opening this issue, the truncate param is not available on the /v1/chat/completions endpoint, this is the case because the /v1/chat/completions aims to follow the same API as openai's chat endpoint.

would it be possible to truncate the input before calling the chat endpoint? If this does not meet your use case we can start a larger conversation about adding this param with the team; however currently we'd like to limit the params to align with openai.

Thank you!