Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NativeDB pruning #1746

Merged
merged 14 commits into from
Feb 4, 2025
Merged

NativeDB pruning #1746

merged 14 commits into from
Feb 4, 2025

Conversation

rakanalh
Copy link
Contributor

@rakanalh rakanalh commented Jan 25, 2025

Description

This PR implements pruning for old native DB records.

When starting at block #1, the version that gets created for all the keys that are stored into the accessory table are assigned 2 which means that the version is ahead of block number by one.

That being said, the current way of pruning is inefficient. Since we have to iterate all entries in the native DB table and compare the version of the key. Unfortunately, there is no way the pruner could figure out which keys specifically changed between two pruning cycles to be able to prune just those.

However, each time the number of entries in this table grows, the pruning process will decrease them significantly which means that the next cycle will need to iterate fewer items.

Linked Issues

@rakanalh rakanalh changed the title Rakanalh/prover storage pruning Prover storage pruning Jan 25, 2025
@rakanalh rakanalh force-pushed the rakanalh/prover-storage-pruning branch from 73a9f44 to 761003d Compare January 30, 2025 11:03
@rakanalh rakanalh marked this pull request as ready for review January 30, 2025 11:12
@rakanalh rakanalh requested a review from eyusufatik as a code owner January 30, 2025 11:12
@auto-assign auto-assign bot requested a review from yaziciahmet January 30, 2025 11:12
@rakanalh rakanalh added the HOLD-MERGE PR is not draft but should not be merged yet label Jan 30, 2025
@rakanalh rakanalh changed the title Prover storage pruning NativeDB pruning Jan 30, 2025
Only enabled for test all flow
Copy link

codecov bot commented Feb 3, 2025

Codecov Report

Attention: Patch coverage is 82.75862% with 15 lines in your changes missing coverage. Please review.

Project coverage is 77.7%. Comparing base (84e51db) to head (a8229bc).
Report is 6 commits behind head on nightly.

Files with missing lines Patch % Lines
...dk/full-node/sov-prover-storage-manager/src/lib.rs 40.0% 9 Missing ⚠️
bin/citrea/src/main.rs 0.0% 3 Missing ⚠️
crates/pruning/src/pruners/native_db.rs 86.3% 3 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
bin/citrea/src/rollup/mod.rs 79.8% <ø> (ø)
crates/fullnode/src/lib.rs 100.0% <100.0%> (ø)
crates/fullnode/src/runner.rs 90.2% <ø> (-0.1%) ⬇️
crates/pruning/src/lib.rs 92.7% <100.0%> (+26.8%) ⬆️
crates/pruning/src/tests.rs 100.0% <100.0%> (ø)
...reign-sdk/full-node/db/sov-db/src/schema/tables.rs 91.5% <ø> (ø)
...overeign-sdk/full-node/db/sov-schema-db/src/lib.rs 77.3% <100.0%> (+0.6%) ⬆️
...sov-prover-storage-manager/src/snapshot_manager.rs 100.0% <100.0%> (ø)
bin/citrea/src/main.rs 0.0% <0.0%> (ø)
crates/pruning/src/pruners/native_db.rs 86.3% <86.3%> (ø)
... and 1 more

... and 17 files with indirect coverage changes

@rakanalh rakanalh requested a review from eyusufatik February 3, 2025 11:18
@rakanalh rakanalh removed the HOLD-MERGE PR is not draft but should not be merged yet label Feb 3, 2025
@rakanalh rakanalh requested a review from eyusufatik February 4, 2025 11:37
@eyusufatik eyusufatik merged commit d2a152f into nightly Feb 4, 2025
14 checks passed
@eyusufatik eyusufatik deleted the rakanalh/prover-storage-pruning branch February 4, 2025 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pruning: ProverStorageManager
2 participants