How can I get real-time outputs from TTS model?

[MekanJuma]

I have been researching this for a long time, but there is no clear and exact answer. I want to be able to get Real-Time results, yeah it depends on where you deployed whether it is GPU or not, but even if I run this in GPU, it takes some time to get output. I would appreciate your help guys!

[CoderCowMoo]

It's extremely difficult to get real-time outputs due to the slow nature of the model. Minimum tinkering approach is to use streaming with @manmay-nakhashi's tortoise-tts-fastest repo, and using streaming output.

To build on that you could try including TensorRT to the codebase (if you do try and get a good implementation and make a pull req).

Further you could try distilling the model, however I wouldn't know where to start.

Good luck