Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have aggregate query scores summed instead of averaged for the gain based metrics #545

Open
david-fisher opened this issue Aug 3, 2022 · 4 comments · May be fixed by #556
Open

Have aggregate query scores summed instead of averaged for the gain based metrics #545

david-fisher opened this issue Aug 3, 2022 · 4 comments · May be fixed by #556

Comments

@david-fisher
Copy link
Contributor

Is your feature request related to a problem? Please describe.
The aggregate score for the set of queries in the case shows the average of the scores for each of the queries. This is not very meaningful for the gain based metrics, CG and DCG.

Describe the solution you'd like
Those two metrics could be summed up, instead of average, giving a clearer picture of the change in the full case when updating judgments on a query.

Describe alternatives you've considered
None
Additional context

@epugh
Copy link
Member

epugh commented Aug 11, 2022

Was moving too fast!

@epugh epugh linked a pull request Aug 27, 2022 that will close this issue
@stale
Copy link

stale bot commented Apr 2, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please do advocate for this issue and ideally submit a patch to get the attention of the maintainers!

@stale stale bot added the wontfix This will not be worked on label Apr 2, 2023
@epugh
Copy link
Member

epugh commented Apr 27, 2023

Going through the new Search Quality class and that reminded me that we really need to get this to fixed. The cobblers children have no shoes ;-(

@stale stale bot removed the wontfix This will not be worked on label Apr 27, 2023
@epugh
Copy link
Member

epugh commented Jun 4, 2023

I really wish we had this........ :-(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants