Bug: setup-default-schema job hangs when Istio sidecar is injected

[JuroOravec]

The setup-default-schema job hangs after completion. I suspect it's because the job has an Istio sidecar injected.

I created TemporalCluster in a namespace with has istio-injection: "enabled" label, which means that Istio sidecars are automatically injected into pods.

apiVersion: v1
kind: Namespace
metadata:
  name: mynamespace
  labels:
    istio-injection: "enabled"

The setup-default-schema did complete (see logs below), but the job still has status COMPLETIONS 0/1, and the job's pod has status NotReady:

NAME                                      COMPLETIONS   DURATION   AGE
job.batch/temporal-setup-default-schema   0/1           21m        21m

NAME                                      READY   STATUS     RESTARTS   AGE
pod/temporal-setup-default-schema-khqfn   1/2     NotReady   2          16m

presenter@Juros-MacBook-Pro helmfile % kubectl -n temporal logs pod/temporal-setup-default-schema-khqfn
2024-04-07T09:37:51.378Z        INFO    Starting schema setup   {"config": {"SchemaFilePath":"","InitialVersion":"0.0","Overwrite":false,"DisableVersioning":false}, "logging-call-at": "setuptask.go:58"}
2024-04-07T09:37:51.378Z        DEBUG   Setting up version tables       {"logging-call-at": "setuptask.go:68"}
2024-04-07T09:37:51.383Z        DEBUG   Setting initial schema version to 0.0   {"logging-call-at": "setuptask.go:119"}
2024-04-07T09:37:51.387Z        DEBUG   Updating schema update log      {"logging-call-at": "setuptask.go:124"}
2024-04-07T09:37:51.390Z        INFO    Schema setup complete   {"logging-call-at": "setuptask.go:132"}

Is there a way to override the annotations for the job's pod template, to disable Istio injection?

[alexandrevilain]

Hi @JuroOravec !

Thanks for reporting this issue.
Did you see that the operator has built-in support for istio mTLS https://temporal-operator.pages.dev/features/mtls/istio/ ?

Job termination is handled and they should not hang indefinitely.