Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/consensus: add new consensus round timer #3440

Merged
merged 32 commits into from
Jan 14, 2025
Merged
Changes from 9 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
eb9c935
add new timer and corresponding feature
DiogoSantoss Jan 7, 2025
5052b77
formatting
DiogoSantoss Jan 7, 2025
834c826
add timer test and fix first round to be one instead of zero
DiogoSantoss Jan 7, 2025
9d0bfd0
fix typo in round timer test
DiogoSantoss Jan 7, 2025
19b8a46
add exponential timer to GetTimerFunc()
DiogoSantoss Jan 7, 2025
d7065be
add consensus round timer documentation
DiogoSantoss Jan 7, 2025
b206e2f
add precedence warning to exponential round timer feature
DiogoSantoss Jan 7, 2025
c8a03d3
add exponential to GetTimerFunc test and fix name consistency
DiogoSantoss Jan 7, 2025
f5ca318
fix lint issue
DiogoSantoss Jan 7, 2025
a4eba85
add same timer warning, cli flag, default timer and more exponential …
DiogoSantoss Jan 8, 2025
0ee9b60
fix default timer doc
DiogoSantoss Jan 8, 2025
1afc273
Update docs/consensus.md
DiogoSantoss Jan 14, 2025
f7da772
Update docs/consensus.md
DiogoSantoss Jan 14, 2025
caec6c3
app/eth2wrap: fallback beacon nodes (#3342)
gsora Jan 7, 2025
e4997eb
*: bump protobuf to 1.36.2 (#3439)
KaloyanTanev Jan 7, 2025
1fdc58f
build(deps): Bump go.opentelemetry.io/otel/trace from 1.32.0 to 1.33.…
dependabot[bot] Jan 7, 2025
bc3c6e7
build(deps): Bump go.opentelemetry.io/contrib/instrumentation/net/htt…
dependabot[bot] Jan 8, 2025
8ce78aa
build(deps): Bump google.golang.org/protobuf from 1.35.2 to 1.36.2 (#…
dependabot[bot] Jan 8, 2025
80d5bf9
build(deps): Bump go.opentelemetry.io/otel/exporters/stdout/stdouttra…
dependabot[bot] Jan 8, 2025
d33769e
build(deps): Bump golang.org/x/tools from 0.28.0 to 0.29.0 (#3443)
dependabot[bot] Jan 8, 2025
66f9e65
app/peerinfo: add nickname field to peerinfo protocol (#3428)
DiogoSantoss Jan 8, 2025
586c265
core/validatorapi: v1/v2 endpoints warning (#3431)
pinebit Jan 8, 2025
1bfee74
app: eth2wrap latency logging (#3417)
KaloyanTanev Jan 8, 2025
bd8efb3
*: bump linter to 1.63.4 (#3444)
KaloyanTanev Jan 8, 2025
7752c0b
build: allow golangci-lint parallel on .pre-commit (#3455)
KaloyanTanev Jan 13, 2025
7109f7e
fix: chown -R runner:docker 001 folder (#3441)
apham0001 Jan 13, 2025
b6841e1
docs: contributing discord channel (#3452)
KaloyanTanev Jan 14, 2025
cf785b5
build(deps): Bump github.com/showwin/speedtest-go from 1.7.9 to 1.7.1…
dependabot[bot] Jan 14, 2025
b8d208d
build(deps): Bump golang.org/x/time from 0.8.0 to 0.9.0 (#3446)
dependabot[bot] Jan 14, 2025
a1985c0
build(deps): Bump sigp/lighthouse from v6.0.0 to v6.0.1 in /testutil/…
dependabot[bot] Jan 14, 2025
9e47a84
build(deps): Bump chainsafe/lodestar from v1.23.1 to v1.24.0 in /test…
dependabot[bot] Jan 14, 2025
7dc6b41
build(deps): Bump github.com/attestantio/go-builder-client from 0.5.2…
dependabot[bot] Jan 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions app/featureset/featureset.go
Original file line number Diff line number Diff line change
@@ -45,6 +45,10 @@ const (
// The feature gets automatically enabled when the current network is gnosis|chiado,
// unless the user disabled this feature explicitly.
GnosisBlockHotfix Feature = "gnosis_block_hotfix"

// Exponential enables Exponential round timer for consensus rounds.
DiogoSantoss marked this conversation as resolved.
Show resolved Hide resolved
// When active has precedence over EagerDoubleLinear round timer.
Exponential Feature = "exponential"
)

var (
@@ -56,6 +60,7 @@ var (
AggSigDBV2: statusAlpha,
JSONRequests: statusAlpha,
GnosisBlockHotfix: statusAlpha,
Exponential: statusAlpha,
// Add all features and there status here.
}

1 change: 1 addition & 0 deletions app/featureset/featureset_internal_test.go
Original file line number Diff line number Diff line change
@@ -16,6 +16,7 @@ func TestAllFeatureStatus(t *testing.T) {
ConsensusParticipate,
JSONRequests,
GnosisBlockHotfix,
Exponential,
}

for _, feature := range features {
46 changes: 46 additions & 0 deletions core/consensus/utils/roundtimer.go
Original file line number Diff line number Diff line change
@@ -3,6 +3,7 @@
package utils

import (
"math"
"strings"
"sync"
"time"
@@ -24,6 +25,12 @@ type TimerFunc func(core.Duty) RoundTimer

// GetTimerFunc returns a timer function based on the enabled features.
func GetTimerFunc() TimerFunc {
if featureset.Enabled(featureset.Exponential) {
return func(core.Duty) RoundTimer {
return NewExponentialRoundTimer()
}
}

if featureset.Enabled(featureset.EagerDoubleLinear) {
return func(core.Duty) RoundTimer {
return NewDoubleEagerLinearRoundTimer()
@@ -47,6 +54,7 @@ func (t TimerType) Eager() bool {
const (
TimerIncreasing TimerType = "inc"
TimerEagerDoubleLinear TimerType = "eager_dlinear"
TimerExponential TimerType = "exponential"
)

// increasingRoundTimeout returns the duration for a round that starts at incRoundStart in round 1
@@ -150,3 +158,41 @@ func (t *doubleEagerLinearRoundTimer) Timer(round int64) (<-chan time.Time, func

return timer.Chan(), func() { timer.Stop() }
}

// exponentialRoundTimer implements a round timerType with the following properties:
//
// The first round has one second to complete consensus
// If this round fails then other peers already had time to fetch proposal and therefore
// won't need as much time to reach a consensus. Therefore start timeout with lower value
// which will increase exponentially
type exponentialRoundTimer struct {
clock clockwork.Clock
}

func (*exponentialRoundTimer) Type() TimerType {
return TimerExponential
}

func (t *exponentialRoundTimer) Timer(round int64) (<-chan time.Time, func()) {
var timer clockwork.Timer
if round == 1 {
// First round has 1 second
timer = t.clock.NewTimer(time.Second)
} else {
// Subsequent rounds have exponentially more time starting at 200 milliseconds
timer = t.clock.NewTimer(time.Millisecond * time.Duration(math.Pow(2, float64(round-1))*100))
}

return timer.Chan(), func() { timer.Stop() }
}

// NewExponentialRoundTimer returns a new exponential round timer type.
func NewExponentialRoundTimer() RoundTimer {
return NewExponentialRoundTimerWithClock(clockwork.NewRealClock())
}

func NewExponentialRoundTimerWithClock(clock clockwork.Clock) RoundTimer {
return &exponentialRoundTimer{
clock: clock,
}
}
59 changes: 59 additions & 0 deletions core/consensus/utils/roundtimer_test.go
Original file line number Diff line number Diff line change
@@ -111,16 +111,75 @@ func TestDoubleEagerLinearRoundTimer(t *testing.T) {
stop()
}

func TestExponentialRoundTimer(t *testing.T) {
tests := []struct {
name string
round int64
want time.Duration
}{
{
name: "round 1",
round: 1,
want: 1000 * time.Millisecond,
},
{
name: "round 2",
round: 2,
want: 200 * time.Millisecond,
},
{
name: "round 3",
round: 3,
want: 400 * time.Millisecond,
},
{
name: "round 4",
round: 4,
want: 800 * time.Millisecond,
},
}

for _, tt := range tests {
fakeClock := clockwork.NewFakeClock()
timer := utils.NewExponentialRoundTimerWithClock(fakeClock)

t.Run(tt.name, func(t *testing.T) {
// Start the timerType
timerC, stop := timer.Timer(tt.round)

// Advance the fake clock
fakeClock.Advance(tt.want)

// Check if the timerType fires
select {
case <-timerC:
default:
require.Fail(t, "Fail", "Timer(round %d) did not fire, want %v", tt.round, tt.want)
}

// Stop the timerType
stop()
})
}
}

func TestGetTimerFunc(t *testing.T) {
timerFunc := utils.GetTimerFunc()
require.Equal(t, utils.TimerEagerDoubleLinear, timerFunc(core.NewAttesterDuty(0)).Type())
require.Equal(t, utils.TimerEagerDoubleLinear, timerFunc(core.NewAttesterDuty(1)).Type())
require.Equal(t, utils.TimerEagerDoubleLinear, timerFunc(core.NewAttesterDuty(2)).Type())

featureset.DisableForT(t, featureset.EagerDoubleLinear)
featureset.EnableForT(t, featureset.Exponential)

timerFunc = utils.GetTimerFunc()
require.Equal(t, utils.TimerExponential, timerFunc(core.NewAttesterDuty(0)).Type())
require.Equal(t, utils.TimerExponential, timerFunc(core.NewAttesterDuty(1)).Type())
require.Equal(t, utils.TimerExponential, timerFunc(core.NewAttesterDuty(2)).Type())

featureset.DisableForT(t, featureset.Exponential)

timerFunc = utils.GetTimerFunc()
require.Equal(t, utils.TimerIncreasing, timerFunc(core.NewAttesterDuty(0)).Type())
require.Equal(t, utils.TimerIncreasing, timerFunc(core.NewAttesterDuty(1)).Type())
require.Equal(t, utils.TimerIncreasing, timerFunc(core.NewAttesterDuty(2)).Type())
28 changes: 22 additions & 6 deletions docs/consensus.md
Original file line number Diff line number Diff line change
@@ -21,19 +21,19 @@ The input to the Priority protocol is a list of protocols defined in order of pr

```json
[
"/charon/consensus/hotstuff/1.0.0", // Highest precedence
"/charon/consensus/abft/2.0.0",
"/charon/consensus/abft/1.0.0",
"/charon/consensus/qbft/2.0.0", // Lowest precedence and the fallback since it is always present
"/charon/consensus/hotstuff/1.0.0", // Highest precedence
"/charon/consensus/abft/2.0.0",
"/charon/consensus/abft/1.0.0",
"/charon/consensus/qbft/2.0.0" // Lowest precedence and the fallback since it is always present
]
```

The output of the Priority protocol is the common "subset" of all inputs, respecting the initial order of precedence, e.g.:

```json
[
"/charon/consensus/abft/1.0.0", // This means the majority of nodes have this protocol available
"/charon/consensus/qbft/2.0.0",
"/charon/consensus/abft/1.0.0", // This means the majority of nodes have this protocol available
"/charon/consensus/qbft/2.0.0"
]
```

@@ -53,6 +53,22 @@ To list all available consensus protocols (with versions), a user can run the co

When a node starts, it sequentially mutates the list of preferred consensus protocols by processing the cluster configuration file and then the mentioned CLI flag. The final list of preferred protocols is then passed to the Priority protocol for cluster-wide consensus. Until the Priority protocol reaches consensus, the cluster will use the default QBFT v2.0 protocol for any duties.

## Consensus Round Duration

There are three diferent round timer implementations. These timers define the duration of each consensus round and how the timing adjusts for subsequent rounds.
pinebit marked this conversation as resolved.
Show resolved Hide resolved

### Increasing Round Timer

The `IncreasingRoundTimer` uses a linear increment strategy for round durations. It starts with a base duration and increases by a fixed increment for each subsequend round. The formula for a given round `n` is `Duration = IncRoundStart + (IncRoundIncrease * n)`.

### Eager Double Linear Round Timer

The `EagerDoubleLinearRoundTimer` aligns start times across participants by starting at an absolute time and doubling the round duration when the leader is active. The round duration increases linearly according to `LinearRoundInc`. This aims to fix an issue with the original solution where the leader resets the timer at the start of the round while others reset when they receive the justified pre-prepare which leads to leaders getting out of sync with the rest.

### Exponential Round Timer
pinebit marked this conversation as resolved.
Show resolved Hide resolved

The `ExponentialRoundTimer` increases round durations exponentially. It provides a sufficient timeout for the initial round and grows from a smaller base timeout for subsequent rounds. The idea behind this timer is that after the first timeout, the remaninig nodes already had time to fetch their proposals and therefore won't need as much time to reach consensus as for the first round.

## Observability

The four existing metrics are reflecting the consensus layer behavior: