FabricDiffusion

Overview

FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Images
Cheng Zhang*, Yuanhao Wang*, Francisco Vicente Carrasco, Chenglei Wu, Jinlong Yang, Thabo Beeler, Fernando De la Torre (* indicates equal contribution)
SIGGRAPH Asia 2024

Updates

[Jan 2 2025] Inference code released.

[Oct 2 2024] Paper released to Arxiv.

Installation

Running the codebase only requires installing a recent version of PyTorch, Diffusers and Transformers:

git clone https://github.com/humansensinglab/fabric-diffusion.git
cd fabric-diffusion
conda env create --file=environment.yml
conda activate fabric-diff

Usage

1. Texture Normalization

Run the following command to normalize texture patches cropped from in-the-wild images:

python inference_texture.py \
    --texture_checkpoint='Yuanhao-Harry-Wang/fabric-diffusion-texture' \
    --src_dir='data/texture_examples' \
    --save_dir='outputs/texture' \
    --n_samples=3

• --texture_checkpoint: path to the pre-trained texture model checkpoint.
• --src_dir: path to the directory containing input images.
• --save_dir: path to the output directory.
• --n_samples: number of samples per input.

2. Print Normalization

Similarly, run the following command to normalize print patches cropped from in-the-wild images:

python inference_print.py \
    --print_checkpoint='Yuanhao-Harry-Wang/fabric-diffusion-print' \
    --src_dir='data/print_examples' \
    --save_dir='outputs/print' \
    --n_samples=3

The model checkpoints are hosted on Huggingface here (texture, print).

We are actively adding more features to this repo. Please stay tuned!

Acknowledgements

• Models
  • Stable Diffusion
  • InstructPix2Pix
  • Matfusion

Citation

If you find this repo useful, please cite:

@inproceedings{zhang2024fabricdiffusion,
    title     = {{FabricDiffusion}: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Images},
    author    = {Zhang, Cheng and Wang, Yuanhao and Vicente Carrasco, Francisco and Wu, Chenglei and 
                 Yang, Jinlong and Beeler, Thabo and De la Torre, Fernando},
    booktitle = {ACM SIGGRAPH Asia},
    year      = {2024},
}

License

We use the X11 License. This license is identical to the MIT License, but with an extra sentence that prohibits using the copyright holders' names (Carnegie Mellon University and Google in our case) for advertising or promotional purposes without written permission.