P2P: Block propagation optimization #1099
Conversation
…and block_notice_message
…de is shutdown right before. This allows stabilization if the bios node is the node that most nodes are getting blocks from.
…ady has. It may have already echoed back the block, but don't want that to prevent the nack from being sent.
…f via last handshake. The last handshake could be very old causing way too many blocks to be sent.
| @@ -193,7 +193,7 @@ def __init__(self, testHelperConfig: TestHelperConfig=TestHelperConfig(), cluste | |||
|
|
|||
| Utils.Debug = self.testHelperConfig.verbose | |||
| self.errorExit = Utils.errorExit | |||
| self.emptyBlockGoal = 1 | |||
| self.emptyBlockGoal = 7 | |||
There was a problem hiding this comment.
This prevents a disconnect from happening right before the performance run. Nicer to have it stable before the performance run.
| @@ -623,6 +628,7 @@ namespace eosio { | |||
| void reset() { | |||
| fc::lock_guard g( _mtx ); | |||
| _write_queue.clear(); | |||
| _sync_write_queue.clear(); | |||
There was a problem hiding this comment.
Added back in the _sync_write_queue removed in #1090. Had a test failure due to block ordering here causing unlinkable blocks. This is really separate from block nack. If desired I could pull it out into a different PR.
| @@ -912,6 +921,13 @@ namespace eosio { | |||
|
|
|||
| std::chrono::nanoseconds connection_start_time{0}; | |||
|
|
|||
| // block nack support | |||
| static constexpr uint16_t consecutive_block_nacks_threshold{2}; // stop sending blocks when reached | |||
There was a problem hiding this comment.
I think this could be as low as 0. 0 worked fine in all my tests and I don't think a large value is useful here.
Also this PR, only doesn't block nack one connection. If we wanted to keep 2 or more peers sending us blocks the PR could be updated with that functionality. I don't personally think it is needed. And I think best to keep this as simple as possible. If we find this feature doesn't work it can be disabled via p2p-disable-block-nack and we can address any issues we find.
|
Note:start |
plugins/net_plugin/include/eosio/net_plugin/auto_bp_peering.hpp
Outdated
Show resolved
Hide resolved
…te determination if a block request is needed. Also include in the block request_message the peer head. The peer head allows the node to determine if on a fork and send from LIB instead. Also create block_on_fork function to remove duplicate code.
Block Nack Feature
Currently when a node has many connections, it can overwhelm its bandwidth when large blocks are created. A node will send every block it receives to all of its peers that it has not already received the block from.
This PR adds a new P2P protocol version, which when both sides of a connection have the feature, it is enabled. For example, if a peer is running
nodeosversion 1.0.x then it will continue to always receive and send all blocks.block_nack_messageandblock_notice_messageto the P2P message protocol.p2p-disable-block-nackwhich disables this functionality. It may be useful to disable on canary/relay nodes of producers so blocks are never missed requiring a block request. Or when absolute block latency desired. Note that latency can be hurt if many peers are configured. For large number of peers it is recommended to not disable the block nack.Algorithm:
block_nack_messagesendsblock_idto peer when it receives a block it has already received from another peer.block_notice_messsagesent instead of a block when a threshold ofblock_nack_messageare received.block_notice_messagewith the existingrequest_messagewhen two consecutive are received for blocks not received from other peers.Resolves #1091