why use different judge model?

[kydxh]

I notice that in run.py, for different datasets, the judge models are different. For example, MCQ datasets use 'chatgpt-0125', and Mathvista uses 'gpt-4-turbo'. Why not use the same judge model for fair comparasion?

[PhoenixZ810]

Hi,

Thank you for your interest and feedback.

It's important to note that the judging model used for each benchmark is aligned with the configurations detailed in the original paper to ensure a fair and consistent comparison. Any changes to the judging model can lead to significantly different results.

We appreciate your understanding on this matter and are open to any further questions or discussions you might have.