Import service-intentions + Separate consul_config_entry for each service-intentions

[DaleyKD]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

Terraform v1.2.6
on windows_amd64
+ provider registry.terraform.io/hashicorp/consul v2.15.1

Affected Resource(s)

Please list the resources as a list, for example:

• consul_config_entry

Terraform Configuration Files

resource "consul_config_entry" "service_intentions" {
  name = "account-api" # Destination
  kind = "service-intentions"

  config_json = jsonencode({
    Sources = [
      {
        Name       = "admin-web"
        Action     = "allow"
      }
    ]
  })
}

Expected Behavior

I have many modules that install their infrastructure separately. They need to be able to configure their own intentions individually.

Module "Account API" should be able to setup its own set of intentions.
Module "Admin Web" should be able to setup its own intentions, separately.

Also, I should be able to import intentions that were previously created.

Actual Behavior

It appears that this resource doesn't allow importing.

Also, the documentation implies that I have to set up all intentions at once and can't do them separately.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

[DaleyKD]

The old consul_intention does exactly what I need.

[DaleyKD]

Upgraded my HCS Cluster today to v1.11.6 (my clients are v1.12.3). I now see that the old consul_intention isn't going to be a great solution. (I'm still waiting? on the old intentions to be migrated to the new service-intentions.)

With that in mind, what are we to do to ensure that we can add these separately and also import them? I COULD use a K8S CRD, but I don't love that.

[remilapeyre]

Hi @DaleyKD, the HTTP API changed regarding the Consul Intentions and the Terraform provider reflect this. I understand that it can be useful to be able to set one intention for a service in a Terraform configuration without changing the other ones. I will look for a solution to support this.

[DaleyKD]

@remilapeyre : Totally understood and greatly appreciated. For the time being, though it's not my preferred method, I will go be using the K8S CRD YAMLs.

[DaleyKD]

@remilapeyre : Went to try and get all the K8S CRDs setup today, and quickly found out that this, too, doesn't work.

When I use Terraform to create a new microservice, I want it to be able to setup the intentions that it needs. This new microservice is the SOURCE, not the destination. However, Consul has it completely inverted: you key off the destination, and it has to know all of its sources.

How can a destination know of its sources if I've never created it before? To me, this seems highly illogical.

Instead of DESTINATION => ["SOURCE1", "SOURCE2", "SOURCE3", "UNKNOWN_SOURCE1", "UNKNOWN_SOURCE2"], it should be SOURCE = ["DESTINATION1", "DESTINATION2", "DESTINATION3"]

Even if Consul doesn't want to go that route, there should at least be a way to add to the source array after the fact.

I'm having to totally change how I do Terraform to handle this.

[remilapeyre]

Hi @DaleyKD, I understand what you mean. The current implementation in the Consul API lets each service control who can access it, which is logical as if the caller and the callee are managed by different teams (they could even be on different Consul cluster) the callee's team should be able to control who can access the service, not the caller's.

I understand that having to manage them from a centralised resource might not be ideal and that you would like to control the access for a specific source and a destination, rather than having to define all sources for a destination at once.

The previous API for Connect intentions used to allow this (https://www.consul.io/api-docs/connect/intentions#create-intention-with-id) but has been deprecated now that intentions are part of config entries.

I will reach out to the Consul team to know if there is any plan to add support for this.

[DaleyKD]

When you write it up like that, explaining the responsibilities of the team, of course it makes sense. 😃 DERP!

But that doesn't fit our company's workflow now. Thanks for your consideration!

[ednxzu]

Hello, I'm having this exact problem right now, thanks for looking into it !
I have multiple repositories to deploy nomad jobs with a home made module that allows teams to start their job, volumes and consul intentions associated with their app.

Currently we're forced to keep using the consul_intention resource, because you cannot use the config_entry resource multiple times for the same service, it'll just override with the latest apply and everything else before that will be gone.

example:
we have promtail logging sidecars in every job, that'll report logs for all the task in the job. Each logging sidecars will need its own intention to be able to talk to the loki backend, but this is not doable atm because everytime you deploy a new config_entry resource pointing to the loki service, it'll override the previous intentions with the new one, denying access to all but the latest sidecar.

[jmnavarrol]

@remilapeyre : Went to try and get all the K8S CRDs setup today, and quickly found out that this, too, doesn't work.

When I use Terraform to create a new microservice, I want it to be able to setup the intentions that it needs. This new microservice is the SOURCE, not the destination. However, Consul has it completely inverted: you key off the destination, and it has to know all of its sources.

How can a destination know of its sources if I've never created it before? To me, this seems highly illogical.

Instead of DESTINATION => ["SOURCE1", "SOURCE2", "SOURCE3", "UNKNOWN_SOURCE1", "UNKNOWN_SOURCE2"], it should be SOURCE = ["DESTINATION1", "DESTINATION2", "DESTINATION3"]

Even if Consul doesn't want to go that route, there should at least be a way to add to the source array after the fact.

I'm having to totally change how I do Terraform to handle this.

As @remilapeyre explained, there is a reason why is the service owner the one allowing access so the expected workflow is for the owner of the calling service to ask permission to the called service's owner who then adds the allowed accesses to the intentions' array.

Not tested, but I don't see why it wouldn't work, what you could do in your scenario is registering your calling services, either on consul itself under a known prefix, or in terraform state for the called service to lookup. That lookup, in turn, builds the origins array.

For a complete solution, "editing" the lookup source should trigger a rebuild of the called service's intentions.

[sorenisanerd]

This seems like something that could be easily handled using Consul's /connect/intention/exact API endpoint, available through the Consul SDK as IntentionUpsert.