Panic when adding secondaryAllocationIds to existing NAT Gateway

[barryrobison]

What happened?

We are attempting to add additional EIPs to our existing NAT Gateways. Adding this causes pulumi to panic during a preview.

  ~ aws:ec2/natGateway:NatGateway: (update)
            [id=nat-0d956f392212314be]
            [urn=urn:pulumi:networking-dev-ap-southeast-2::networking::networking$aws:ec2/natGateway:NatGateway::main-vpc-nat-1]
            [provider=urn:pulumi:networking-dev-ap-southeast-2::networking::pulumi:providers:aws::networking-dev-main::b2a1187a-d9f4-4a79-a67b-f5abc56a5db7]
          ~ secondaryAllocationIds: [
              + [0]: "eipalloc-05f44fxxxxx"
            ]
panic: fatal: An assertion has failed: Expected diff to not require deletion or replacement during Update of urn:pulumi:networking-dev-ap-southeast-2::networking::networking$aws:ec2/natGateway:NatGateway::main-vpc-nat-1

This operation should not cause a deletion or replacement of an existing NAT GW.

Example

new aws.ec2.NatGateway(
                `${name}-vpc-nat-${i}`,
                {
                  allocationId: elasticIPs[i].allocationId,
                  secondaryAllocationIds: ["eipalloc-05f44f1e5xxxxx"],
                  subnetId,
                  tags,
                },
                { provider, parent: this },
              )

Output of pulumi about

CLI          
Version      3.86.0
Go Version   go1.21.1
Go Compiler  gc

Plugins
NAME    VERSION
aws     6.8.0
docker  3.6.1
nodejs  unknown

Host     
OS       debian
Version  11.7
Arch     aarch64

This project is written in nodejs: executable='/usr/local/bin/node' version='v18.18.0'

Current Stack: networking-dev-ap-southeast-2

TYPE                                                 URN
pulumi:pulumi:Stack                                  urn:pulumi:networking-dev-ap-southeast-2::networking::pulumi:pulumi:Stack::networking-networking-dev-ap-southeast-2

Found no pending operations associated with networking-dev-ap-southeast-2

Backend        
Name           993e6d61911f
URL            s3://xxxxx/
User           root
Organizations  
Token type     personal

Dependencies:
NAME                              VERSION
@pulumi/aws                       6.8.0
@pulumi/awsx                      0.40.1
@pulumi/pulumi                    3.93.0
axios                             1.6.0
eslint                            8.52.0
eslint-plugin-import              2.29.0
handlebars                        4.7.8
lint-staged                       15.0.2
typescript                        5.2.2
@types/node                       20.8.10
@typescript-eslint/eslint-plugin  6.9.1
@typescript-eslint/parser         6.9.1
eslint-config-prettier            9.0.0
prettier                          3.0.3

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[mikhailshilkov]

Thank you for reporting this issue @barryrobison

No root cause yet, but quick notes so far: the error is coming from TF bridge Update: https://github.com/pulumi/pulumi-terraform-bridge/blob/f217804896b3dfa0c48627686de58f5e624a9c18/pkg/tfbridge/provider.go#L980

The code hasn't changed for a long while. A similar issue that was closed in the past: pulumi/pulumi-terraform-bridge#296

[mikhailshilkov]

@barryrobison Can you describe which other resources I need to create to repro the issue? How is elasticIPs defined? Any chance for a standalone repro that I could just run locally?

[barryrobison]

@barryrobison Can you describe which other resources I need to create to repro the issue? How is elasticIPs defined? Any chance for a standalone repro that I could just run locally?

Hi Mikhail - here is a minimal repro:

https://github.com/barryrobison/pulumi-issue-repro/tree/main/aws-2971

You'll need to provide your own subnet and EIP allocation ids but hopefully this is sufficient. Thanks!

[mikhailshilkov]

Thank you @barryrobison. I got the issue reproduced with the following isolated program:

import * as aws from "@pulumi/aws";

const defaultVpc = aws.ec2.getVpcOutput({default: true});
const defaultVpcSubnets = aws.ec2.getSubnetsOutput({
    filters: [
        {name: "vpc-id", values: [defaultVpc.id]},
    ],
});

const one = new aws.ec2.Eip("one", {
  domain: "vpc",
});

const two = new aws.ec2.Eip("two", {
  domain: "vpc",
});

new aws.ec2.NatGateway("ng",
  {
      allocationId: one.allocationId,
      // after a first run to create the NAT Gateway, uncomment
      // the following line to reproduce a terraform panic
      // secondaryAllocationIds: [two.allocationId],
      subnetId: defaultVpcSubnets.ids[0],
  },
);

Then, on the second update I get the same panic.

[iwahbe]

Digging in, I have confirmed that this is a provider bug. @barryrobison Thanks for raising the issue and @mikhailshilkov thanks for building the self-contained repro. It made chasing this down much faster.

---

This is another bug caused by makeDetailedDiff missing computed inputs, in this case that a new computed input requires a replace. As such, the fix will come in a qualified bridge release.

[barryrobison]

This is another bug caused by makeDetailedDiff missing computed inputs, in this case that a new computed input requires a replace. As such, the fix will come in a qualified bridge release.

Thanks Ian and Mikhail!

Just to clarify, adding the Secondary EIPs to an existing NAT Gateway should not cause a replace.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-natgateway.html

I can't find anything definitive in the Terraform docs about how it handles a change of this property..

Thanks!

[iwahbe]

I don't see it documented anywhere, but TF handles it the same way that we do, with a replace. I checked by translating the Pulumi program into HCL: https://gist.github.com/iwahbe/d11ab2a962a69394741150e8ce0e7694.

[t0yv0]

Fixed in v6.9.0