Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-claim deleted vector spaced in vec0 chunks more comprehensively #184

Open
asg017 opened this issue Jan 23, 2025 · 0 comments
Open

Re-claim deleted vector spaced in vec0 chunks more comprehensively #184

asg017 opened this issue Jan 23, 2025 · 0 comments

Comments

@asg017
Copy link
Owner

asg017 commented Jan 23, 2025

If you delete a row from a vec0 table, then the vector's space in the chunk will be "empty". If you insert a new row right after, sqlite-vec will only re-claim the space if the latest available chunk has an open source (ie if the deleted vector was in the last ~1024 inserted vectors).

We should make it so that when a chunk is full, we check previous chunks to see if there are empty slots, and insert there instead.

Doing this in a performant way will be difficult, it may make INSERTs on large tables less performant. Maybe a vec_popcount() SQL functions could make this easier? idk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant