Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Results reproducability #4

Open
Kuzaiherba opened this issue Aug 2, 2024 · 0 comments
Open

Results reproducability #4

Kuzaiherba opened this issue Aug 2, 2024 · 0 comments

Comments

@Kuzaiherba
Copy link

Kuzaiherba commented Aug 2, 2024

Hello,
thanks for your paper and congratulations on the publication in Nature!

I was trying to reproduce your PPI model's results on the STRING dataset but was unable to reach the reported performance for both PPI-residue-importance (for 1UZN) and best-F1 score (best-F1 achieved is 0.86 vs. reported 0.95).

A few caveats - I used the published random split for training; and to extract residue importance, I used a torch's GNNExplainer customised to be compatible with ProteinGNNTrainer. The .pdb files were obtained using AF2 (which is expected to produce lower performance according to the results you reported for folded & actual structures).

Would you mind sharing the weights of the PPI model or maybe adding extra details relevant to reproduce the reported results (e.g., the exact splits you used, and how residue importance was extracted)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant