BucketListDB Random Eviction Cache

[SirTyson]

Description

Resolves #3696

Adds a random eviction cache to LiveBucket using DiskIndex. Currently, I've set the default to cache up to 25% of Bucket entries, which puts total stellar-core memory usage at about 5-6 GB by default.

Checklist

• Reviewed the contributing document
• Rebased on top of master (no merge commits)
• Ran clang-format v8.0.0 (via make format or the Visual Studio extension)
• Compiles
• Ran all tests
• If change impacts performance, include supporting evidence per the performance document

[dmkozh]

I wonder how is one supposed to reason about this value. Are the RAM requirements roughly BL size * cached percent? Maybe it's worth adding a comment regarding this.
I'm also curious why do we use percentage here instead of e.g. the number of entries or even the rough RAM boundary.

[SirTyson]

Yeah, usage is basically BL size * cached percentage. The reason I did percentage is primarily for easier implementation. Max number of items to cache is a poor metric, since we now have high variance in LedgerEntry size because of Soroban entries.

RAM based caps are more useful, but pose some implementation challenges. For example, suppose we need to cache a large WASM entry. If we use a random eviction cache, we might need to evict many entries to free up enough space for the WASM entry. It is also a little complicated to determine how you allocate RAM among the different buckets. Suppose you have a limit of 1 GB, how do we divy this up among the different buckets? You could do something like each bucket gets a cache size relative to how much of the total BucketList size the bucket makes up. However, we now have to initialize caches wrt total BucketList size, which is challenging since Bucket objects are created individually in a background thread well before they are required.

All of these are probably solvable issues, but I doubt anyone will actually modify this value. In practice we can probably manage this value ourselves as needed by changing the default value in point release should bucketList increase significantly. In practice this ratio is just the upper bound. Especially for smaller buckets, I imagine they will churn before filling up their cache anyway.

[dmkozh]

Max number of items to cache is a poor metric, since we now have high variance in LedgerEntry size because of Soroban entries.

But this value is set based on the entry count as well, isn't it? I just think it's much easier to argue about the RAM usage when you deal with a constant number of entries - given that this number is large enough and given the random eviction safe, I think it's safe to just multiply this by the mean entry size to get the RAM estimate. While I agree that this is unlikely to be changed, I find it hard to argue about this value even for ourselves, especially if BL grows and we just forget about this.

[dmkozh]

Forgot to approve this; I think we can change the config semantics later if we want. This still needs to be rebased though.