feat(mempool): mempool transaction limits, saving to db and continue block prod from pending block

[cchudant]

Pull Request type

• Feature

Add limits to the mempool:

• transaction age
• max number of transactions in the mempool
• additional limit of declare transactions in the mempool

fixes #373

[cchudant]

Added:

• Mempool transaction are now saved to and recovered from DB
• Block production can now continue the current pending block (fixes bug: pending txs should be stored in graceful shutdown #373)
• Added e2e tests that restart madara devnet to test these two features.

[shamsasari]

This is quite a large PR. Can it not be broken down, especially if there's two features?

[cchudant]

This is quite a large PR. Can it not be broken down, especially if there's two features?

i am thinking about it
i think it's just easier at this point to have it as one pull request, all of the changes here are in the mempool.

I believe if i were to split this pr in 2 or 3, one of these pr would necessarily still have >50 files changed.. that would be the one fixing saving the txs to the db, it's kind of a pain to split that because it touches all of the mempool crate, adds a db columns, and refactors all of the to_blockifier stuff

[cchudant]

the mempool limits and continue block prod from pending block should only change a few files really, it's the mempool saving that's making it complex

[cchudant]

@shamsasari i did the fixes you mentioned, except the splitting the pr one

[Trantorian1]

A few minute points might need changing. This is really cool work! (Also please for the love of god and everything dear do not force push, I don't want to have to sift through all these files again next time I review 😅 )

[Trantorian1]

All good! Great work 🔥