Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow fixing the p2p IP #29

Open
PierreBesson opened this issue Dec 17, 2021 · 4 comments
Open

Allow fixing the p2p IP #29

PierreBesson opened this issue Dec 17, 2021 · 4 comments

Comments

@PierreBesson
Copy link
Contributor

PierreBesson commented Dec 17, 2021

Automatic node port discovery was introduced in #28, however in some cases the operator will not want / be able to open a large range of ports (30000-32767) on their Kubernetes nodes.

An option should be added to fix the attributed nodePort, however in this case it might be impossible to support more than 1 replica for the statefulset as fixing the port will result in a port conflict for the second replica.

Similarly, it should be possible to deploy a node which uses a fixed p2p IP by using loadBalancerIP for LoadBalancer services using a pre-reserved IP at the Cloud Provider (eg. for GCP).
However in this case, the p2p service would no longer be of type NodePort but instead of type LoadBalancer.

@PierreBesson PierreBesson changed the title Allow fixing the p2p port Allow fixing the p2p port and IP Jan 3, 2022
@dblane-digicatapult
Copy link
Contributor

When using type LoadBalancer our implementation would vary depending on the cloudProvider being used when we query a service to get the external address which is either an FQDN or an IP, which we then need to advertise when using --public-addr

AWS example:

status:
  loadBalancer:
    ingress:
    - hostname: some-loadbalancer-hostname.eu-west-1.elb.amazonaws.com

Where as GCP will return:

status:
    loadBalancer:
      ingress:
      - ip: 111.111.111.111

This complicates how we then advertise, do we want to detect the address's format and then setup our multiaddr format based on if its an IP or FQDN? The current kubectl image we are using for the init container that queries these services has no dig or hosts command so reducing an FQDN to either a single or an array of IPs is challenging.

Also currently our naming is very specific to NodePort node.perNodeServices.createP2pNodePortService Do we want to change something more generically named for the p2p service? E.G.

  perNodeServices:
    createApiService: true
    createP2pService: true
    p2pServiceType: LoadBalancer  # Must be type ClusterIP, NodePort or LoadBalancer
    setPublicAddressToExternal:
      enabled: true
      ipRetrievalServiceUrl: https://ifconfig.io/ip

@dblane-digicatapult
Copy link
Contributor

@PierreBesson Any comments on above suggestions?

@PierreBesson
Copy link
Contributor Author

PierreBesson commented Mar 1, 2022

For me the renaming you propose withcreateP2pService and p2pServiceType make sense. As for supporting FQDN instead of IPs for --public-addr, I didn't even know this was possible but I would be fine to add this feature if it is strictly additive to regular IP-based public address.
That would mean maybe setting up a different init-container for doing FQDN resolution when the option is enabled.

@PierreBesson
Copy link
Contributor Author

Fixing the p2p ports is now possible with chart v2.0 using the node.perNodeServices.relayP2pService.port and node.perNodeServices.paraP2pService.port properties. However fixing the public IP with the Service loadBalancerIP still needs to be implemented.

@PierreBesson PierreBesson changed the title Allow fixing the p2p port and IP Allow fixing the p2p ~port and~ IP May 12, 2022
@PierreBesson PierreBesson changed the title Allow fixing the p2p ~port and~ IP Allow fixing the p2p IP May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants