Adding Multi-Image generation usecase app #3356
Conversation
There was a problem hiding this comment.
Overall looks good to me,
I think the intro section needs more highlighting of what is the differentiated solution here vs using Stable diffusion. There is a lot happening under the hood, which needs to be shown & explained.
Also please check the changes in the docs folder in this PR
You need to add this to make it show up in https://pytorch.org/serve/genai_use_cases.html
examples/usecases/llm_diffusion_serving_app/docker/client_app.py
Outdated
Show resolved
Hide resolved
examples/usecases/llm_diffusion_serving_app/docker/client_app.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Ankith Gunapal <[email protected]>
examples/usecases/llm_diffusion_serving_app/docker/sd/stable_diffusion_handler.py
Show resolved
Hide resolved
examples/usecases/llm_diffusion_serving_app/docker/client_app.py
Outdated
Show resolved
Hide resolved
| print(f"\nResults and Images saved at {out_dir} which is a Docker container mount, corresponds to 'serve/model-store-local/' on the host machine.\n") | ||
|
|
||
| if __name__ == "__main__": | ||
| main() |
There was a problem hiding this comment.
Very cool and comprehensive script for benchmarking!
| logger.info("Compiling UNet.") | ||
| if compile_mode == "max-autotune" and change_comp_config: | ||
| pipe.unet.to(memory_format=torch.channels_last) | ||
| torch._inductor.config.conv_1x1_as_mm = True |
There was a problem hiding this comment.
What are these configs? Are these missing in the benchmarking script?
There was a problem hiding this comment.
These settings provide better perf on GPUs.
These settings are not used in our app.
In SD model-config.yaml for this app, we set change_comp_config=False.
There was a problem hiding this comment.
Oh I see. In the blog , we can add a note about these configs for GPU and what benefit they provide
examples/usecases/llm_diffusion_serving_app/docker/client_app.py
Outdated
Show resolved
Hide resolved
Description
This Streamlit app is designed to generate multiple images based on a provided text prompt. It leverages TorchServe for efficient model serving and management, and utilizes Meta-LLaMA-3.2 for prompt generation, and Stable Diffusion with latent-consistency/lcm-sdxl and Torch.compile using OpenVINO backend for image generation.