Skip to content

Experimental features

Jump to bottom
Woolverine94 edited this page Feb 1, 2024 · 14 revisions

ROCm support under GNU/Linux

  • As of commit 7c88732, biniou introduce experimental ROCm support under GNU/Linux, via the update_rocm.sh script :

  • Complementary prerequisites are a 4GB+ VRAM AMD GPU (or iGPU if eligible) using a working ROCm 5.6 environment and an already functional biniou standard installation.

  • You can easily activate ROCm support by selecting the type of optimization to activate (CPU, CUDA or ROCm for Linux), in the WebUI control module.

Note: I don't have access to AMD compatible hardware to validate that the modifications for the ROCm compatibility are working. Any feedback through a discussion (if working and give a significant boost to inferences duration) or an issue ticket (if not working or broken), with details on your setup and tested modules will be greatly appreciated 🙏.

Following content is outdated , please click here

CUDA support

  • biniou has been thinked as a cpu-only-no-gpu-required application, but it should be really easy to make it use your NVIDIA GPU to accelerate inferences. As of commit [630c975] (11/27/23), biniou will support the following experimental features for all modules, except Chatbot and Faceswap modules as a test :
    • Autodetection of CUDA device and configuration of biniou to use it
    • If CUDA is enabled, using fp16 torch_dtype, which will force you to re-download the models, but half the size of them (when supported).
    • If CUDA is enabled, using cpu_offload to save as much VRAM as possible (when supported).
  • Complementary prerequisites are a 4GB+ VRAM Nvidia GPU using a working CUDA 12.1 environment and an already functional biniou standard installation.
  • As biniou use the cpu-only PyTorch version, you must replace it with the default one to benefits from these changes :

On Debian 12, it could be done with something like that :

cd ./biniou
source ./env/bin/activate
pip uninstall torch torchvision torchaudio
pip install torch==2.1.0 torchvision torchaudio # Re-installing the default cuda-enabled version.
deactivate
./webui.sh

Rollback if not working :

cd ./biniou
source ./env/bin/activate
pip uninstall torch torchvision torchaudio
pip install torch==2.1.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu # Re-installing the cpu-only version.
deactivate
./webui.sh

On Windows 10/11, using CMD :

cd "%userprofile%\biniou"
call venv.cmd
pip uninstall torch torchvision torchaudio 
pip install torch==2.1.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Rollback if not working :

cd "%userprofile%\biniou"
call venv.cmd
pip uninstall torch torchvision torchaudio 
pip install torch==2.1.0 torchvision torchaudio

Note: I don't have access to CUDA compatible hardware to validate that the modifications for the CUDA compatibility are working. Thanks to @koinkoin-project tests, we already validated that the detection of CUDA works, but biniou can't generate contents with only 2GB VRAM. Any feedback through a discussion (if working and give a significant boost to inferences duration) or an issue ticket (if not working or broken), with details on your setup and tested modules will be greatly appreciated 🙏.

Clone this wiki locally