New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix mistral attention on Metal #2699

Merged

LaurentMazare merged 1 commit into huggingface:main from bancek:fix-mistral-metal

Jan 4, 2025

+2 −1

Contributor

bancek commented Jan 4, 2025

This PR fixes Mistral attention on Metal device.

Before fix:

Metal(KernelError(MatMulNonContiguous { lhs_stride: [64, 4, 2, 1], rhs_stride: [2048, 64, 1024, 1], mnk: (2, 64, 2) }))


          Fix mistral attention on Metal

5ee6dc7

LaurentMazare approved these changes

View reviewed changes

LaurentMazare merged commit 57f41da into huggingface:main

10 checks passed

Collaborator

LaurentMazare commented Jan 4, 2025

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet