v0.9.59.rc0

What's Changed

• Client side validation for non fp8 kv cache and fp8 context fmha by @joostinyi in #1302
• Add encoder model support in trt-llm by @michaelfeil in #1294
• Add kv_cache_host_memory_bytes as a configurable runtime setting by @joostinyi in #1303

Full Changelog: v0.9.58...v0.9.59.rc0

v0.9.58

What's Changed

• Update trt_llm_config.py by @pankajroark in #1291
• Add DRAFT_EXTERNAL as a default speculator.speculative_decoding_mode by @joostinyi in #1292
• LazyDataResolver falls back to data dir if no space in cache by @helenlyang in #1293
• Prep for Smoke Tests by @marius-baseten in #1296
• Chains Smoke Tests by @marius-baseten in #1298
• Upgrade CI ubuntu runners to 22.04. by @marius-baseten in #1299
• Sync poetry and python versions with CI by @marius-baseten in #1300
• Release 0.9.58 by @basetenbot in #1301

Full Changelog: v0.9.57...v0.9.58

v0.9.58rc102

What's Changed

• LazyDataResolver falls back to data dir if no space in cache by @helenlyang in #1293
• Prep for Smoke Tests by @marius-baseten in #1296
• Chains Smoke Tests by @marius-baseten in #1298
• Upgrade CI ubuntu runners to 22.04. by @marius-baseten in #1299
• Sync poetry and python versions with CI by @marius-baseten in #1300

Full Changelog: v0.9.58rc2...v0.9.58rc102

v0.9.58rc2

What's Changed

• Add DRAFT_EXTERNAL as a default speculator.speculative_decoding_mode by @joostinyi in #1292

Full Changelog: v0.9.58rc1...v0.9.58rc2

v0.9.58rc1

What's Changed

• Update trt_llm_config.py by @pankajroark in #1291

Full Changelog: v0.9.57...v0.9.58rc1

v0.9.57

What's Changed

• Make readiness_endpoint liveness_endpoint required to use custom server by @tianshuc0731 in #1267
• send truss version on patch by @rcano-baseten in #1268
• Speculative Decoding Interface refactor by @joostinyi in #1270
• Update trt_llm_config.py to add encoder by @michaelfeil in #1274
• Update trt_llm_config.py -> revision by @michaelfeil in #1269
• Better chains error propagation (+various fixes). by @marius-baseten in #1271
• Bump briton in truss library by @joostinyi in #1273
• Support package patches in build from dir by @marius-baseten in #1275
• Selective watch, fixes BT-12924 by @marius-baseten in #1278
• Automatic migration of TRTLLM runtime configuration by @joostinyi in #1279
• fix initialization from pydantic models by @joostinyi in #1281
• Update Chains Docs. by @marius-baseten in #1282
• Various. Fixes BT-12926,BT-10647,BT-12585 by @marius-baseten in #1284
• bump briton package to 0.3.13.dev3 by @joostinyi in #1286
• Release 0.9.57 by @basetenbot in #1287

New Contributors

• @michaelfeil made their first contribution in #1274

Full Changelog: v0.9.56...v0.9.57

v0.9.57rc1

What's Changed

• Update Chains Docs. by @marius-baseten in #1282
• Various. Fixes BT-12926,BT-10647,BT-12585 by @marius-baseten in #1284
• bump briton package to 0.3.13.dev3 by @joostinyi in #1286

Full Changelog: v0.9.56rc3...v0.9.57rc1

v0.9.56rc3

What's Changed

• fix initialization from pydantic models by @joostinyi in #1281

Full Changelog: v0.9.56rc2...v0.9.56rc3

v0.9.56rc2

What's Changed

• Support package patches in build from dir by @marius-baseten in #1275
• Release 0.9.56 by @basetenbot in #1276
• Selective watch, fixes BT-12924 by @marius-baseten in #1278
• Automatic migration of TRTLLM runtime configuration by @joostinyi in #1279

Full Changelog: v0.9.56rc1...v0.9.56rc2

v0.9.56

What's Changed

• Release 0.9.56 by @basetenbot in #1276

Full Changelog: v0.9.55...v0.9.56