torch 2.4.0 does not work with fp8 training

[ddpasa]

The readme on the sd3 branch recommends installing torch==2.4.0 and torchvision==0.19.0. However, torch 2.4.0 gives the following error with flux fp8 training:

AttributeError: module 'torch' has no attribute 'float8_e4m3fnuz'

Upgrading to torch 2.4.1 fixes this issue. However, torchvision 0.19.0 is not compatible with 2.4.1 if you try to pip install, because torchvision 0.19.0 wants torch 2.4.0.

I have a hacky workaround where I install torch 2.4.0 to make torchvision 0.19.0 happy, and then I upgrade torch to 2.4.1. It seems to work fine, but there is probably a better solution :)

[ddpasa]

update, this fixed my issue, seems like a clean solution:

pip install torch==2.4.1 torchvision xformers --index-url https://download.pytorch.org/whl/cu124