Fix nodes not getting set to offline when disconnects #2131
Conversation
Signed-off-by: Kristoffer Dalby <[email protected]>
WalkthroughThe pull request introduces enhancements to the integration testing framework by adding a new test case, Changes
Assessment against linked issues
Possibly related PRs
Poem
Tip OpenAI O1 model for chat
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range and nitpick comments (1)
integration/route_test.go (1)
1327-1328: Please address the commented-out line.Either uncomment the line
defer scenario.ShutdownAssertNoPanics(t)if it's required for the test, or remove it entirely to keep the code clean.
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (3)
- .github/workflows/test-integration.yaml (1 hunks)
- integration/route_test.go (2 hunks)
- integration/scenario.go (1 hunks)
Additional comments not posted (3)
.github/workflows/test-integration.yaml (1)
61-61: Excellent addition of a new test case!The new test case
TestHASubnetRouterFailoverWhenNodeDisconnects2129is a great addition to improve the test coverage for the scenario described in issue #2129. The test name clearly describes the scenario it is testing and follows the naming convention of other test cases. It is added to the appropriate place in the existing list of integration tests.No other changes are required in this file to support the addition of this test case.
integration/scenario.go (1)
653-665: LGTM!The
DisconnectContainersfunction is a valuable addition to theScenariostruct. It provides the ability to selectively disconnect containers from the network, enhancing the control and flexibility of the integration tests.The implementation is sound, with proper error handling and checks for container existence. This function will be useful for simulating network disruptions and testing the system's behaviour under such conditions.
Great work on improving the test suite's capabilities!
integration/route_test.go (1)
1319-1517: The test implementation looks good!The test effectively verifies the failover behaviour of HA subnet routers when a node disconnects. It sets up the scenario correctly, advertises and enables routes properly, simulates the node disconnection realistically, and makes relevant assertions at each stage. The test structure and flow are clear and comprehensive.
Signed-off-by: Kristoffer Dalby <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
Outside diff range and nitpick comments (2)
integration/scenario.go (2)
656-662: Consider logging a warning if a container is not found in the pool.The function correctly handles the case where a container might not exist in the pool by using the comma ok idiom. However, it might be beneficial to log a warning in such cases, as it could indicate a potential issue in the test setup.
Apply this diff to log a warning:
if ctr, ok := s.pool.ContainerByName(container); ok { err := ctr.DisconnectFromNetwork(s.network) if err != nil { return err } +} else { + log.Printf("Warning: Container %s not found in the pool", container) }
670-676: Consider logging a warning if a container is not found in the pool.Similar to the
DisconnectContainersFromScenariofunction, it might be beneficial to log a warning if a container is not found in the pool, as it could indicate a potential issue in the test setup.Apply this diff to log a warning:
if ctr, ok := s.pool.ContainerByName(container); ok { err := ctr.ConnectToNetwork(s.network) if err != nil { return err } +} else { + log.Printf("Warning: Container %s not found in the pool", container) }
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (3)
- hscontrol/poll.go (4 hunks)
- integration/route_test.go (2 hunks)
- integration/scenario.go (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- integration/route_test.go
Additional comments not posted (4)
hscontrol/poll.go (4)
194-194: Capturing the start time is a good addition for logging purposes.Introducing the
startvariable to record the start time of the long polling operation enables measuring the duration of various events within the function. This will provide valuable insights into the performance and timing of the long polling process, aiding in monitoring and debugging.
252-252: Including the duration in the cancellation log message adds valuable context.Logging the time elapsed since the start of the long polling operation when a poll is cancelled provides additional insights into when the cancellation occurred. This information can be useful for understanding the timing and behaviour of poll cancellations, aiding in debugging and performance analysis.
256-257: Enhancing the context done log message with duration and error details improves debugging.Logging the time elapsed since the start of the long polling operation and the associated error message when the poll context is done provides a more complete picture of the event. This additional information can greatly assist in understanding the timing and reason behind the context completion, making it easier to debug issues and monitor the system's behaviour.
348-348: Simplifying response handling and improving error messages enhances code clarity.The changes made to directly use the
http.Flusherinterface for flushing responses, instead of relying on aResponseController, streamline the response handling flow. This simplification makes the code easier to understand and maintain.Additionally, the slight rephrasing of error messages related to keep-alive improves their clarity, making it easier for developers to comprehend and troubleshoot any issues that may arise.
Overall, these modifications enhance the code's readability and maintainability without altering the core functionality.
Also applies to: 362-372
| // ConnectContainersToScenario disconnects a list of containers from the network. | ||
| func (s *Scenario) ConnectContainersToScenario(containers ...string) error { | ||
| for _, container := range containers { | ||
| if ctr, ok := s.pool.ContainerByName(container); ok { | ||
| err := ctr.ConnectToNetwork(s.network) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return nil | ||
| } |
There was a problem hiding this comment.
Consider extracting the common logic into a separate function to make the code more DRY.
The ConnectContainersToScenario and DisconnectContainersFromScenario functions share the same structure and logic, with the only difference being the specific action performed on the container (connect or disconnect). To make the code more DRY (Don't Repeat Yourself), consider extracting the common logic into a separate function that takes a function parameter for the specific action.
Here's an example of how the refactored code could look:
func (s *Scenario) performActionOnContainers(containers []string, action func(*dockertest.Container) error) error {
for _, container := range containers {
if ctr, ok := s.pool.ContainerByName(container); ok {
err := action(ctr)
if err != nil {
return err
}
} else {
log.Printf("Warning: Container %s not found in the pool", container)
}
}
return nil
}
func (s *Scenario) DisconnectContainersFromScenario(containers ...string) error {
return s.performActionOnContainers(containers, func(ctr *dockertest.Container) error {
return ctr.DisconnectFromNetwork(s.network)
})
}
func (s *Scenario) ConnectContainersToScenario(containers ...string) error {
return s.performActionOnContainers(containers, func(ctr *dockertest.Container) error {
return ctr.ConnectToNetwork(s.network)
})
}
|
The work missing here is that we cannot in go reliable determine when a node disconnects, this is handled further down the stack. To resolve the linked issue, and have this test passing, we need to implement something that routinely "pings" any router, or at least routers that are determined to be HA. This can probably be implemented with something like PingRequest(https://github.com/tailscale/tailscale/blob/main/tailcfg/tailcfg.go#L1717) |
Fixes #2129
Summary by CodeRabbit
New Features
Bug Fixes