env vars vs. bootstrapConfig

[klemen-df]

Hi,

I've read that bootstrapConfig is only allowed to be configured when creating an EMQX cluster, and does not support updating. The bootstrap config can only be set at creation time or modified via the dashboard.

Also, found this : 'Configuration priority: emqx.conf < ENV < cluster-overrride.conf' - not really well documented, btw :)

Anyhow, what about scenario where I don't have any bootstrapConfig? Where all of the configuration is an environment variable?

What happens if I create a cluster (for example) with:
EMQX_LISTENER_SSL_OPTS__VERSIONS=["tlsv1.3","tlsv1.2","tlsv1.1"]

and then I change the CRD and (try to) apply it with:
EMQX_LISTENER_SSL_OPTS__VERSIONS=["tlsv1.3","tlsv1.2"]

(variable being - listener_ssl_opts.versions, here)

Will it (should it) be applied?

[Rory-Z]

Hi, @KlemenDanfoss

Anyhow, what about scenario where I don't have any bootstrapConfig? Where all of the configuration is an environment variable?

If you don't set any bootstrapConfig, the EMQX Operator will add some default config for EMQX custom resource, you can check it by kubectl get emqx emqx -o json | jq .spec.bootstrapConfig after deploy EMQX.

What happens if I create a cluster (for example) with:
EMQX_LISTENER_SSL_OPTS__VERSIONS=["tlsv1.3","tlsv1.2","tlsv1.1"]
and then I change the CRD and (try to) apply it with:
EMQX_LISTENER_SSL_OPTS__VERSIONS=["tlsv1.3","tlsv1.2"]

EMQX Operator will not enable SSL listener by default, so I'm not sure what happened, @zhongwencool could you please take a look ?

[zhongwencool]

It's useless for you to EMQX_LISTENER_SSL_OPTS__VERSIONS=["tlsv1.3","tlsv1.2"].
If you don't want to change via bootstrapConfig,
You have to write the entire SSL listener configuration to the environment variable.
EMQX_LISTENER_SSL_DEFAULT={ bind = "0.0.0.0:8883" max_connections = 512000 ssl_options { keyfile = "etc/certs/key.pem" certfile = "etc/certs/cert.pem" cacertfile = "etc/certs/cacert.pem" } }

[klemen-df]

So, scenario where it works:

• nothing set (empty) in bootstrapConfig
• EMQX_LISTENERS__SSL__MQTTS__BIND="0.0.0.0:8883"
• EMQX_LISTENERS__SSL__MQTTS__SSL_OPTIONS__VERSIONS=["tlsv1.3","tlsv1.2"]
• EMQX_LISTENERS__SSL__MQTTS__SSL_OPTIONS__KEYFILE="/folder/keyfile.pem"
• EMQX_LISTENERS__SSL__MQTTS__SSL_OPTIONS__CERTFILE="/folder/certfile.pem"
• EMQX_LISTENERS__SSL__MQTTS__SSL_OPTIONS__CACERTFILE="/folder/cacertfile.pem"

[klemen-df]

Hi,

thanks for the clarification. Probably this should be documented better.

But in our use-case - we use Terraform/IaaC - we don't make any changes through the dashboard so env vars are fine.

[Rory-Z]

Very cool !

[alexa22013]

Hey, @Rory-Z as you mentioned

Yes, the bootstrapConfig can not change after the initial run, but environment vars can be changed.

But as per the github documentation https://github.com/emqx/emqx-operator/blob/main/docs/en_US/reference/v2alpha1-reference.md#:~:text=List%20of%20environment%20variables%20to%20set%20in%20the%20container.%20Cannot%20be%20updated envs cannot be updated

So is it possible to update config dynamically or atleast are a rolling update in gke cluster, and also without using the dashboard?

[klemen-df]

I changed tls settings (just the env var) and it worked.

[Rory-Z]

@alexa22013

But as per the github documentation https://github.com/emqx/emqx-operator/blob/main/docs/en_US/reference/v2alpha1-reference.md#:~:text=List%20of%20environment%20variables%20to%20set%20in%20the%20container.%20Cannot%20be%20updated envs cannot be updated

This "cannot be updated" means if update env vars, the container will restart. So you can update env vars, and wait EMQX containers rolling updated