-
Notifications
You must be signed in to change notification settings - Fork 766
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
litep2p: Provide partial results to speedup GetRecord queries #7099
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
@@ -850,7 +850,7 @@ linked-hash-map = { version = "0.5.4" } | |||
linked_hash_set = { version = "0.1.4" } | |||
linregress = { version = "0.5.1" } | |||
lite-json = { version = "0.2.0", default-features = false } | |||
litep2p = { version = "0.8.4", features = ["websocket"] } | |||
litep2p = { git = "https://github.com/paritytech/litep2p.git", branch = "lexnv/kad-found-records", features = ["websocket"] } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Release is needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, will do one soon, wanted to double-check first on our Kusama validator that this indeed solves the problem 🙏
CPULitep2p without this PR was running before ~7:15 (spike—left side). On the right side is Litep2p with this PR. The CPU consumption looks overall stable, with some fluctuations at ~10-minute intervals. Considering the network's randomness and the ~18x speedup improvement in authority-discovery records, this small tradeoff (if any) appears justified. Triage ReportThe
|
This PR provides the partial results of the
GetRecord
kademlia query.This significantly improves the authority discovery records, from ~37 minutes to ~2/3 minutes.
In contrast, libp2p discovers authority records in around ~10 minutes.
The authority discovery was slow because litep2p provided the records only after the Kademlia query was completed. A normal Kademlia query completes in around 40 seconds to a few minutes.
In this PR, partial records are provided as soon as they are discovered from the network.
Testing Done
Started a node in Kusama with
--validator
and litep2p backend.The node discovered 996/1000 authority records in ~ 1 minute 45 seconds.
Before & After
In this image, on the left side is libp2p, in the middle litep2p without this PR, on the right litep2p with this PR
Closes: #7077
cc @paritytech/networking