Conflict with dataset updates

[angelavargas00]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.2.9

• provider registry.terraform.io/hashicorp/google v6.8
• provider registry.terraform.io/hashicorp/google-beta v6.8

Affected Resource(s)

google_bigquery_dataset

Terraform Configuration

Bigquery Module

terraform {
  required_version = ">= 0.15"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 6.0"
    }
    google-beta = {
      source  = "hashicorp/google-beta"
      version = "~> 6.0"
    }
  }
  experiments = [module_variable_optional_attrs]
}


resource "google_bigquery_dataset" "default" {
  project       = var.project_id
  dataset_id    = local.dataset_id
  friendly_name = "testing_01" -> "testing_02"
  description   = var.description
  location      = var.location

  labels = merge(
    var.tags,
    {
      "resource_name"   = local.dataset_id,
      "resource_type"   = "bq",
      "deployment_date" = formatdate("DD-MM-YYYY", timestamp())
    }
  )
  }

App configuration

# resource "google_bigquery_dataset_iam_member" "raw_bq_viewer_prev" {

  #  project    = local.datalake_project_raw
  #  dataset_id = module.bigquery_raw["raw1"].dataset_id
  #  role       = "roles/bigquery.metadataViewer"
  #  member     = local.prev_group
#  }

Debug Output

module.data-factory.module.bigquery[0].google_bigquery_dataset.default will be updated in-place

 ~ resource "google_bigquery_dataset" "default" {

     ~ friendly_name                   = "testing_01" -> "testing_02"

   }

module.data-factory.google_bigquery_dataset_iam_member.raw_bq_viewer_prev[0] will be destroyed



module.data-factory.module.bigquery[0].google_bigquery_dataset.default: Modifications complete after 1s



module.data-factory.google_bigquery_dataset_iam_member.raw_bq_viewer_prev[0]: Still destroying... [10s elapsed]

module.data-factory.google_bigquery_dataset_iam_member.raw_bq_viewer_prev[0]: Destruction complete after 13s

Expected Behavior

The iam member should be removed and the dataset friendly_name should be updated.

Actual Behavior

However, the iam member persists in the dataset configuration after the Terraform execution. Analyzing the GCP Logging, we can observe that immediately after the deletion of the iam member, there is another addition of the same resource.

The explanation of this behaviour: If we update the dataset (in any of their variables) at the same time we add/remove iam members on this dataset, Terraform gets a conflict and the deletion is not working.

Steps to reproduce

1. terraform apply

Important Factoids

No response

References

No response

[ggtisc]

Hi @angelavargas00

I tried to replicate this issue with the following steps:

1. Create the google_bigquery_dataset and google_bigquery_dataset_iam_member resources
2. Destroy the google_bigquery_dataset_iam_member resource
3. Change the google_bigquery_dataset.friendly_name from testing_01 to testing_02

But after looking in the tfstate file the value was correctly changed and no errors happened during these steps

This is the used code, you could check it and see if you have something similar:

resource "google_bigquery_dataset" "bq_dataset_20995" {
  project       = "project-20995"
  dataset_id    = "bq_dataset_20995"
  friendly_name = "bq-ds-20995"
  description   = "something"
  location      = "us-central1"

  labels = {
    "resource_name"   = "bq-dataset-20995"
    "resource_type"   = "bq"
    "deployment_date" = formatdate("DD-MM-YYYY", timestamp())
  }
}

resource "google_bigquery_dataset_iam_member" "bq_ds_iam_member_20995" {
  project    = "project-20995"
  dataset_id = google_bigquery_dataset.bq_dataset_20995.dataset_id
  role       = "roles/bigquery.metadataViewer"
  member     = "user:[email protected]"
}

I suggest you try again without using variables and modules to detect what is happening in your environment and then ensure your module configuration and variables are correct. If after this you are still having issues try with basic troubleshooting restarting everything on your environment

[angelavargas00]

Hi @ggtisc,

The point is to make these changes at the same time:

1. Create the google_bigquery_dataset and google_bigquery_dataset_iam_member resources: OK
2. Destroy the google_bigquery_dataset_iam_member resource and change the google_bigquery_dataset.friendly_name from testing_01 to testing_02 -> In the same change

As I posted, the Terraform output shows both changes, but finally it generates a conflict and the final tfstate does not show the deletion of iam members.

[angelavargas00]

Hi @ggtisc,

An additional point, the Google Support Team gave us this explanation:

"We have inspected the logs which you have provided and could see the permissions got added back because Terraform updated the google_bigquery_datasets with the old access lists after the corresponding iam_member was deleted.
Based on the timestamps in the logs, destruction of _raw_bq_viewer_prev was applied first. Then, based on the plan preview, Terraform want to update the dataset. And when it did try to do an update, it used an access list that still had the old permissions."

Thanks in advance

[ggtisc]

Thanks @angelavargas00 that helps so much

As the message suggests and according to the shared error message, this is orchestrated by your module configuration, Which means this is a mistake in the configuration but not a bug. I suggest you check that configuration, because terraform by itself doesn't inherently support applying such changes within a single configuration. However, you can achieve this using a two-step approach:

1. Destroy the google_bigquery_dataset_iam_member resource:
   This can be done by adding a separate Terraform configuration file (or within the same file if preferred) to manage the lifecycle of the IAM member. In this configuration, define the google_bigquery_dataset_iam_member resource with a lifecycle block specifying destroy_after_deployment = true. This ensures the resource is removed after the initial deployment.

2. Update the friendly_name of google_bigquery_dataset

By applying these configurations sequentially (destroy the IAM member first, then update the dataset name), you can achieve the intended outcome. In a resume this is a bad configuration, not a bug because resources works as expected with the standard usage, for more information you could read the official documentation to know how Google Cloud services (link here) and terraform resources works (link here)

Alternatively you can do it with sample commands or with a 3rd party script, but we cannot take actions in these cases because it is out of terraform scope and the configuration of the modules depends on users. In any case both