Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rotary embeddings #321

Open
wsnoble opened this issue Apr 17, 2024 · 0 comments
Open

Add rotary embeddings #321

wsnoble opened this issue Apr 17, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@wsnoble
Copy link
Contributor

wsnoble commented Apr 17, 2024

Apparently in NLP, rotary embeddings are being used somewhat frequently. Rather than absolute positional information, they capture relative positional information. In mass spec, differences between peaks are also important. Would require changes in DepthCharge to do this. Will says there is an option to use a custom function for the encoding. We would just need to write the code to do the rotary embeddings.

Here is a description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant