Skip to content

Fix wrong token latency when batch size is greater than 1 #3525

Fix wrong token latency when batch size is greater than 1

Fix wrong token latency when batch size is greater than 1 #3525