Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: 'Adapter' #444

Open
FunkyFrog1 opened this issue Nov 13, 2024 · 1 comment
Open

KeyError: 'Adapter' #444

FunkyFrog1 opened this issue Nov 13, 2024 · 1 comment

Comments

@FunkyFrog1
Copy link

Error while loading adapter
It seems that i used wrong ckpt, but I've try
'./IP-Adapter/sdxl_models/ip-adapter_sdxl.bin' './IP-Adapter/sdxl_models/ip-adapter_sdxl.safetensors' './IP-Adapter/sdxl_models/ip-adapter_sdxl_vit-h.bin' './IP-Adapter/sdxl_models/ip-adapter_sdxl_vit-h.safetensors'
And It didn't work

Loading pipeline components...: 100%|██████████| 7/7 [00:00<00:00, 8.14it/s]
Traceback (most recent call last):
File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/SDXL_Adapter.py", line 43, in
main()
File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/SDXL_Adapter.py", line 29, in main
ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, 'cuda')
File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/Adapter/ip_adapter.py", line 84, in init
self.load_ip_adapter()
File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/Adapter/ip_adapter.py", line 137, in load_ip_adapter
ip_layers.load_state_dict(state_dict["Adapter"])
KeyError: 'Adapter'

And here is my code

pipe = load_pipe()
image_encoder_path = './IP-Adapter/sdxl_models/image_encoder'
ip_ckpt = './IP-Adapter/sdxl_models/ip-adapter_sdxl.bin'
ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, 'cuda')

I refer to the demo

And I download the whole files from here

My packages are:
diffusers==0.28.0 torch==2.3.0 tokenizers==0.19.1 transformers==4.41.1

@FunkyFrog1
Copy link
Author

def load_ip_adapter(self):
    if os.path.splitext(self.ip_ckpt)[-1] == ".safetensors":
        state_dict = {"image_proj": {}, "Adapter": {}}
        with safe_open(self.ip_ckpt, framework="pt", device="cpu") as f:
            for key in f.keys():
                print(key)
                if key.startswith("image_proj."):
                    state_dict["image_proj"][key.replace("image_proj.", "")] = f.get_tensor(key)
                elif key.startswith("ip_adapter."):
                    state_dict["Adapter"][key.replace("ip_adapter.", "")] = f.get_tensor(key)
    else:
        state_dict = torch.load(self.ip_ckpt, map_location="cpu")
    self.image_proj_model.load_state_dict(state_dict["image_proj"])
    ip_layers = torch.nn.ModuleList(self.pipe.unet.attn_processors.values())
    ip_layers.load_state_dict(state_dict["Adapter"])

Use safetensors format and replace load_ip_adapter() function in Class IPAdapter may works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant