Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducing SMURF baseline numbers on TAP-Vid DAVIS #1

Open
SeKim12 opened this issue Sep 23, 2024 · 0 comments
Open

Reproducing SMURF baseline numbers on TAP-Vid DAVIS #1

SeKim12 opened this issue Sep 23, 2024 · 0 comments

Comments

@SeKim12
Copy link

SeKim12 commented Sep 23, 2024

Hi, thank you for introducing this method and the implementation! I had a quick question regarding the table in your paper with TAP-Vid metrics for different baseline models (page 5, Table 1). Using the official SMURF implementation and provided checkpoint, I am unable to reproduce the numbers that were reported here. Here is a side-by-side comparison

Metrics Paper My code
AJ 29.8 25.7
AD 27.4 32.4
Avg pts 42.7 37.3

I am resizing all DAVIS frames (and labels) to (256, 256) when computing the metric. I am also using the sintel-smurf checkpoint. It would be immensely helpful if you could point me to the specific configurations or code that you used to generate the numbers reported in the paper. I found the results to be quite interesting, especially because I could not find any other papers evaluating smurf on Tap-Vid DAVIS, and would love to look into this further. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant