turbomind 支持结构化输出 #3047
Comments
|
我们调研下 mlc-ai/xgrammar |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
{{ message }}
|
我们调研下 mlc-ai/xgrammar |
Motivation
目前lmdeploy pytorch engine已经支持结构化输出，但pytorch engine 速度确实比turbomind慢很多，而且输出的延迟也比较大，特别是首次输出。所以还是希望turbomind engine能够更快的支持 结构化输出，12月以来开源社区guided decoding发展迅速，outlines-dev/outlines, mlc-ai/xgrammar,，noamgat/lm-format-enforcer，都有很好的效果，特别是mlc-ai/xgrammar非常好。感谢贵团队的开源工作！
Related resources
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: