Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make scann range search results more accurate #179

Closed
wants to merge 1 commit into from
Closed

Make scann range search results more accurate #179

wants to merge 1 commit into from

Conversation

chasingegg
Copy link
Collaborator

@chasingegg chasingegg commented Nov 6, 2023 •
edited
Loading

Related #171
Range search's recall of ScaNN is poor due to quantization. Enlarge radius to take more candidates into consideration.

Before change

SCANN {"metric_type": "L2", "radius": 65.0} 0.0088
SCANN {"metric_type": "IP", "radius": 8.699999809265137} 0.003
SCANN {"metric_type": "COSINE", "radius": 0.20000000298023224} 0.0018

After

SCANN {"metric_type": "L2", "radius": 65.0} 0.2133
SCANN {"metric_type": "IP", "radius": 8.699999809265137} 0.0339
SCANN {"metric_type": "COSINE", "radius": 0.20000000298023224} 0.0272

@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chasingegg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mergify Mergify
Copy link

mergify bot commented Nov 6, 2023

@chasingegg 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

@chasingegg
Copy link
Collaborator Author

/hold

@mergify mergify bot added the ci-passed label Nov 6, 2023
alexanderguzhva
alexanderguzhva requested changes Nov 6, 2023
View reviewed changes
thirdparty/faiss/faiss/IndexIVFPQFastScan.cpp
@@ -1332,7 +1332,8 @@ void IndexIVFPQFastScan::range_search_implem_12(
TIC;

// prepare the result handlers
std::unique_ptr<RangeSearchResultHandler<C, true>> handler(new RangeSearchResultHandler<C, true>(result, radius, 0, bitset));
float radius_thresh = C::is_max ? radius * 1.2 : radius * 0.6; // to make range search result more accurate
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such a change should never be done on the Faiss side, but on the Knowhere side. Please move it to ivf.cc.

@chasingegg
Copy link
Collaborator Author

close this, need further investigation

@chasingegg chasingegg closed this Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexanderguzhva alexanderguzhva alexanderguzhva requested changes

@hhy3 hhy3 Awaiting requested review from hhy3

@zhengbuqian zhengbuqian Awaiting requested review from zhengbuqian

Assignees
No one assigned
Labels
approved ci-passed dco-passed do-not-merge/hold do-not-merge/missing-related-issue size/XS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@chasingegg @sre-ci-robot @alexanderguzhva