AMD Compatibility

[latchk3y]

While obviously not a priority feature, opening the project up to a broader spectrum of GPUs would benefit long term generation and development.

[neonbjb]

Do it and submit a PR! It shouldn't be hard. I know a few people have gotten it working.

[latchk3y]

it's good to know that it's possible, but would you mind pointing me in the direction of how it could be done? every guide and set-up i've seen thus far appears to be running it from CUDA.

[neonbjb]

I don't know how, as I have never trained or done inference on an AMD card! I'd suggest researching how to get pytorch models to run on rocm cards. There's probably a few good guides on doing it with stable diffusion out there. If you can get stable diffusion running, tortoise will likely run in the same environment.

[Clybius]

Works with ROCm on Ubuntu 20.04 over here.

All you need is the ROCm version of Torch and of course a ROCm installation on your system.

I setup a venv (python3 -m venv venv) and did the setup in the readme from there, with --extra-index-url https://download.pytorch.org/whl/rocm5.2.
Inside the venv, python -m pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/rocm5.2 should get you most of the way there.
It took a bit of fiddling around with numpy/numba (had to just pip install numpy rather than relying on the version in requriements.txt for example)

You may need to run python with these environment args, replacing the versions with what is appropriate for your GPU: HSA_OVERRIDE_GFX_VERSION=10.3.0 PYTORCH_ROCM_ARCH="gfx1034" (this is for an RX 6400, refer to rocminfo and use a bit of Google-Fu for your specific GPU if you're still unsure.)

[latchk3y]

while this is good information to have for Linux distributions, it appears that such a feat is not possible on windows, because as far as i can tell, windows cannot run rocm without a lot of configuration and dependency nightmare.

hopefully i or someone else can find a work around for this soon.

[MTSMATT]

do you fix it?

[fakerybakery]

Works on AMD for me. On Linux + PyTorch + ROCm, it treats AMD cards as CUDA. Existing code should work

[Chapoly1305]

Manage to make it works on Windows with a 6800XT.

#501 (comment)

[Everan82]

Would that work on a 6700xt? How did you managed to make it work?

Manage to make it works on Windows with a 6800XT.

#501 (comment)

[Chapoly1305]

Would that work on a 6700xt? How did you managed to make it work?

Manage to make it works on Windows with a 6800XT.

#501 (comment)

I have tested with 6800XT and consider 6700XT is extremely close.
I assume it will, you can give it a try with a smaller batch size base on the VRAM.

[Everan82]

I feel stupid, it only runs on CPU :(

…

On Sun, 7 Jan 2024, 15:45 Super Chiba, ***@***.***> wrote: Would that work on a 6700xt? How did you managed to make it work? Manage to make it works on Windows with a 6800XT. #501 (comment) <#501 (comment)> I have tested with 6800XT and consider 6700XT is extremely close. I assume it will, you can give it a try with a smaller batch size base on the VRAM. — Reply to this email directly, view it on GitHub <#284 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AV7HCYSUDSC7JVAU2EHBZA3YNK7JJAVCNFSM6AAAAAAUNVIFEWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DAMZXHA3TC> . You are receiving this because you commented.Message ID: ***@***.***>

[Chapoly1305]

You can try this fork for using a AMD GPU under Windows.

https://github.com/Chapoly1305/tortoise-tts-directml

[Everan82]

Will try thanks a lot

…

On Sun, 7 Jan 2024, 15:50 Super Chiba, ***@***.***> wrote: I feel stupid, it only runs on CPU :( On Sun, 7 Jan 2024, 15:45 Super Chiba, *@*.***> wrote: Would that work on a 6700xt? How did you managed to make it work? Manage to make it works on Windows with a 6800XT. #501 <#501> (comment) #501 (comment) <#501 (comment)> I have tested with 6800XT and consider 6700XT is extremely close. I assume it will, you can give it a try with a smaller batch size base on the VRAM. — Reply to this email directly, view it on GitHub #284 (reply in thread) <#284 (reply in thread)> , or unsubscribe https://github.com/notifications/unsubscribe-auth/AV7HCYSUDSC7JVAU2EHBZA3YNK7JJAVCNFSM6AAAAAAUNVIFEWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DAMZXHA3TC . You are receiving this because you commented.Message ID: *@*.***> You can try this fork for using a AMD GPU under Windows. https://github.com/Chapoly1305/tortoise-tts-directml — Reply to this email directly, view it on GitHub <#284 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AV7HCYU7NYTXW4E4HLMESCLYNK74ZAVCNFSM6AAAAAAUNVIFEWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DAMZXHEZTI> . You are receiving this because you commented.Message ID: ***@***.***>