Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Set serviceName on StatefulSets #152

Merged
merged 1 commit into from
Nov 21, 2024
Merged

fix: Set serviceName on StatefulSets #152

merged 1 commit into from
Nov 21, 2024

Conversation

mhutter
Copy link
Member

@mhutter mhutter commented Nov 15, 2024

In order for pods in a STS to "know" their FQDN (instead of just their hostname), the spec.serviceName attribute must be set. This will configure their FQDN to be
${metadata.name}.${spec.serviceName}.${metadata.namespace}.svc.cluster.local

One example where this is needed is when deploying a MongoDB Replicaset; Pods can only communicate with each other via the FQDN, not their hostnames alone; but they won't recognize "themselves" in the member list when they aren't aware of their own FQDN.

Checklist

  • Categorize the PR by setting a good title and adding one of the labels:
    bug, enhancement, documentation, change, breaking, dependency
    as they show up in the changelog
  • Update tests.

In order for pods in a STS to "know" their FQDN (instead of just their
hostname), the `spec.serviceName` attribute must be set. This will
configure their FQDN to be
`${metadata.name}.${spec.serviceName}.${metadata.namespace}.svc.cluster.local`

One example where this is needed is when deploying a MongoDB Replicaset;
Pods can only communicate with each other via the FQDN, not their
hostnames alone; but they won't recognize "themselves" in the member
list when they aren't aware of their own FQDN.

Signed-off-by: Manuel Hutter <[email protected]>
@mhutter mhutter added the bug Something isn't working label Nov 15, 2024
@mhutter mhutter requested a review from davidgubler November 15, 2024 10:10
@mhutter mhutter self-assigned this Nov 15, 2024
@martini-source
Copy link
Member

LGTM, thank you!

@mhutter mhutter merged commit 88b796f into master Nov 21, 2024
4 checks passed
@mhutter mhutter deleted the sts-servicename branch November 21, 2024 13:51
@davidgubler
Copy link
Member

This change is breaking existing setups, and I think it should be reverted. This also needs a migration path.

The reason is that this tries to change the "serviceName" field of existing statefulSets, but that's not allowed: "updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"

I don't have a good idea yet how to fix this...

@davidgubler
Copy link
Member

One way to fix this which worked for me: Delete the StatefulSet while keeping the PV, and let k8ify re-create the statefulSet.

@davidgubler
Copy link
Member

davidgubler commented Dec 5, 2024

The proper way to fix this problem is to use

kubectl -n <namespace> delete statefulset <name> --cascade=orphan

Note --cascade=orphan which will ensure that the pods associated with the StatefulSet that you’re deleting continue running.

Redeploy using k8ify immediately after, otherwise you might get issues later.

Since this problem hasn't been addressed for two weeks now it probably doesn't make much sense anymore to revert the change. Instead unfortunately users will have to modify their StatefulSets using the mentioned procedure (which would have been necesssary anyway, but it should have been documented better).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants