Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not store writes in the snapshot #151

Merged
merged 1 commit into from
Dec 18, 2024
Merged

Do not store writes in the snapshot #151

merged 1 commit into from
Dec 18, 2024

Conversation

gsserge
Copy link
Contributor

@gsserge gsserge commented Dec 15, 2024

  • Do not store writes in both the snapshot and the write set. Storing them is definitely convenient, because then all kinds of gets and scans become very easy to implement. Unfortunately, this convenience comes at a great cost, both in CPU and memory, because the writes to CoW Vart tree are expensive. Our benchmarks have demonstrated about 20% reduction in memory usage and speed of inserts.
  • Change scan(), scan_at_ts() and get_at_ts() to consider both the write set and the snapshot of a transaction. For the scans, the actual complexity is implemented in a new helper method merging_scan.
  • Since our improved Snapshot Isolation does not need to track reads anymore, this PR also disables storing reads and scans by default, thus saving even more memory.

@arriqaaq I think we need to normalize all types of scans in vart:

  • make them all have the same return type, e.g. impl Iterator<Item = (&[u8], &[u8], u64, u64)>;
  • do not materialize scans in Vec's right away, always return an iterator.

The fact that currently range() and scan_at_ts() return considerably different types has made it very difficult to factor out the common merging logic into Transaction::merging_scan. Regular structures always lead to a simpler code.

Benchmarks

  1. 1 million inserts (surrealdb/surrealdb@main...sergii/mem_bench_1mil), both mem and speed improved
  • on main
mem_1mil   fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ runner  50.29 s       │ 50.29 s       │ 50.29 s       │ 50.29 s       │ 1       │ 1
           max alloc:    │               │               │               │         │
             25784649    │ 25784649      │ 25784649      │ 25784649      │         │
             2.513 GB    │ 2.513 GB      │ 2.513 GB      │ 2.513 GB      │         │
           alloc:        │               │               │               │         │
             179605744   │ 179605744     │ 179605744     │ 179605744     │         │
             44.62 GB    │ 44.62 GB      │ 44.62 GB      │ 44.62 GB      │         │
           dealloc:      │               │               │               │         │
             175605454   │ 175605454     │ 175605454     │ 175605454     │         │
             49.07 GB    │ 49.07 GB      │ 49.07 GB      │ 49.07 GB      │         │
           grow:         │               │               │               │         │
             13001236    │ 13001236      │ 13001236      │ 13001236      │         │
             4.839 GB    │ 4.839 GB      │ 4.839 GB      │ 4.839 GB      │         │
           shrink:       │               │               │               │         │
             102         │ 102           │ 102           │ 102           │         │
             532 B       │ 532 B         │ 532 B         │ 532 B         │         │

  • this PR
mem_1mil   fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ runner  38.61 s       │ 38.61 s       │ 38.61 s       │ 38.61 s       │ 1       │ 1
           max alloc:    │               │               │               │         │
             18133827    │ 18133827      │ 18133827      │ 18133827      │         │
             1.968 GB    │ 1.968 GB      │ 1.968 GB      │ 1.968 GB      │         │
           alloc:        │               │               │               │         │
             134841584   │ 134841584     │ 134841584     │ 134841584     │         │
             40.88 GB    │ 40.88 GB      │ 40.88 GB      │ 40.88 GB      │         │
           dealloc:      │               │               │               │         │
             131841308   │ 131841308     │ 131841308     │ 131841308     │         │
             45.3 GB     │ 45.3 GB       │ 45.3 GB       │ 45.3 GB       │         │
           grow:         │               │               │               │         │
             13001213    │ 13001213      │ 13001213      │ 13001213      │         │
             4.797 GB    │ 4.797 GB      │ 4.797 GB      │ 4.797 GB      │         │
           shrink:       │               │               │               │         │
             102         │ 102           │ 102           │ 102           │         │
             596 B       │ 596 B         │ 596 B         │ 596 B         │         │
  1. Range scan bench from SurrealKV, important because of the merging scan and not tracking reads by default for SI
$ cargo bench --bench store_bench range_scan
   Compiling surrealkv v0.6.1 (/home/ubs/p/surrealkv)
    Finished `bench` profile [optimized] target(s) in 5.15s
     Running benches/store_bench.rs (target/release/deps/store_bench-a4d9983290826233)
Gnuplot not found, using plotters backend
range_scans/range_scan/100
                        time:   [11.561 µs 11.565 µs 11.570 µs]
                        change: [-15.638% -15.581% -15.521%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) low severe
  3 (3.00%) low mild
  1 (1.00%) high mild
  3 (3.00%) high severe
range_scans/range_scan/1000
                        time:   [114.66 µs 114.69 µs 114.73 µs]
                        change: [-17.295% -17.145% -17.023%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
Benchmarking range_scans/range_scan/10000: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.7s, enable flat sampling, or reduce sample count to 60.
range_scans/range_scan/10000
                        time:   [1.3155 ms 1.3160 ms 1.3166 ms]
                        change: [-19.588% -19.487% -19.387%] (p = 0.00 < 0.05)
                        Performance has improved.

@gsserge gsserge requested a review from arriqaaq December 15, 2024 14:23
@arriqaaq
Copy link
Contributor

@gsserge Can we attach the benchmark results just to measure the difference/impact this PR makes compared to main?

@arriqaaq arriqaaq merged commit 4fcf5b2 into main Dec 18, 2024
7 checks passed
@arriqaaq arriqaaq deleted the snapshot_writes branch December 18, 2024 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants