Bug with gavinbunney/terraform-provider-kubectl + k8s 1.27

[zack-is-cool]

Persona

delivery iac unicorn

Description

there's a bug in the kubectl terraform provider. seems like it's unable to see things that are already existing or unable to put them into state or something. The thing gets made, but subsequent runs it tries to make the thing again. Specifically seeing this with the vpc-cni eniconfigs in our eks module.

gavinbunney/kubectl provider used in vpc-cni-custom-networking is behaving inconsistently · aws-ia/terraform-aws-eks-blueprints
aws-ia/terraform-aws-eks-blueprints#1675

failed to fetch resource from kubernetes: the server could not find the requested resource · gavinbunney/terraform-provider-kubectl
gavinbunney/terraform-provider-kubectl#270

there are some solutions in the threads above.

1. use null_provider + bash script

• (pls no)

2. use a "raw" helm chart method

• <name> failed to fetch resource from kubernetes: the server could not find the requested resource gavinbunney/terraform-provider-kubectl#270 (comment)

3. use https://github.com/alekc/terraform-provider-kubectl

• might work.. looks maintained? Easy drop-in solution

Impact

it's bad

Completion

• Migrate away from using gavinbunney/terraform-provider-kubectl provider as it's not maintained
• Establish new pattern for applying "IaC level" kubernetes manifests in https://github.com/defenseunicorns/terraform-aws-uds-eks

[zack-is-cool]

Spoke with @ntwkninja.

Conclusion:
Use helm_release resource from the hashicorp/helm provider with a "raw" helm chart to create arbitrary kubernetes resources. Meaning we will feed arbitrary manifests into it. Likely, we will create and publish our own raw helm chart.

[zack-is-cool]

working this here
https://github.com/defenseunicorns/terraform-aws-uds-eks/pull/53

you can apply eniconfigs through the addon:

aws eks describe-addon-configuration \
    --addon-name vpc-cni \
    --addon-version v1.14.0-eksbuild.3 \
    --query 'configurationSchema' \ 
    --output text | jq