Bigquery Dataset panic on partial state resource .get due to TF Importer method not being called

[guineveresaenger]

What happened?

We have a recurring panic fixed via patch, where if we are reading a gcp.bigquery.dataset resource with a nonexistent Project, the provider panics on preview.

This panic should instead surface an error.

We did not find a corresponding issue in the upstream provider and therefore suspect this is a bridge bug; however, we should verify that this is not an upstream bug.

Example

Here's the program (must be run on unpatched GCP provider, e.g. v6.50.0)

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

// Get a dataset

const dataset = gcp.bigquery.Dataset.get(
    'salesforce',
    'salesforce',
    {
        project: 'bc-prod-fusion',
    },
    {},
);

dataset.id.apply((id) => {
    console.log(`got dataset ${id}`);
});

export const selflink = dataset.selfLink

and after pulumi up we get the following stack trace:

 error: preview failed

    panic: interface conversion: interface {} is nil, not string
    goroutine 156 [running]:
    github.com/hashicorp/terraform-provider-google-beta/google-beta/services/bigquery.resourceBigQueryDatasetRead(0xc00680e400, {0x4fcf7a0?, 0xc0064c3500})
    	/Users/guin/go/src/github.com/pulumi/pulumi-gcp/upstream/google-beta/services/bigquery/resource_bigquery_dataset.go:685 +0x1450
    github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x58120b0?, {0x58120b0?, 0x9066720?}, 0xd?, {0x4fcf7a0?, 0xc0064c3500?})
    	/Users/guin/go/pkg/mod/github.com/pulumi/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:788 +0x163
    github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc00082d260, {0x58120b0, 0x9066720}, 0xc0015c8dd0, {0x4fcf7a0, 0xc0064c3500})
    	/Users/guin/go/pkg/mod/github.com/pulumi/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1094 +0x552
    github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfshim/sdk-v2.v2Provider.Refresh({0xc0009ad0e0?, {0x0?, 0x0?, 0x0?}}, {0x504cf3e, 0x17}, {0x58125f8?, 0xc006673f98}, {0x57ead00, 0xc00677cfc0})
    	/Users/guin/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/[email protected]/pkg/tfshim/sdk-v2/provider.go:124 +0x1f1
    github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfbridge.(*Provider).Read(0xc000193080, {0x5811fd0?, 0xc00677ccf0?}, 0xc005fa3ce0)
    	/Users/guin/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/[email protected]/pkg/tfbridge/provider.go:851 +0x715
    github.com/pulumi/pulumi-terraform-bridge/x/muxer.(*muxer).Read.func1({0x5829470?, 0xc000193080?})
    	/Users/guin/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/x/[email protected]/muxer.go:350 +0x33
    github.com/pulumi/pulumi-terraform-bridge/x/muxer.resourceMethod[...](0xc005c0a280?, 0x40, 0xc005eb7778?)
    	/Users/guin/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/x/[email protected]/muxer.go:303 +0xb5
    github.com/pulumi/pulumi-terraform-bridge/x/muxer.(*muxer).Read(0x0?, {0x5811fd0?, 0xc00677ccf0?}, 0x40?)
    	/Users/guin/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/x/[email protected]/muxer.go:349 +0x65
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Read_Handler.func1({0x5811fd0, 0xc00677ccf0}, {0x4cbda00?, 0xc005fa3ce0})
    	/Users/guin/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/proto/go/provider_grpc.pb.go:591 +0x75
    github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x5811fd0, 0xc00677c9f0}, {0x4cbda00, 0xc005fa3ce0}, 0xc00673b3e0, 0xc006673f20)
    	/Users/guin/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/go/otgrpc/server.go:57 +0x3d0
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Read_Handler({0x4f82c80?, 0xc005c0a280}, {0x5811fd0, 0xc00677c9f0}, 0xc00675e690, 0xc00068f320)
    	/Users/guin/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/proto/go/provider_grpc.pb.go:593 +0x135
    google.golang.org/grpc.(*Server).processUnaryRPC(0xc000f90000, {0x58243e0, 0xc004de9ba0}, 0xc006678b40, 0xc005c50930, 0x901d258, 0x0)
    	/Users/guin/go/pkg/mod/google.golang.org/[email protected]/server.go:1360 +0xe15
    google.golang.org/grpc.(*Server).handleStream(0xc000f90000, {0x58243e0, 0xc004de9ba0}, 0xc006678b40, 0x0)
    	/Users/guin/go/pkg/mod/google.golang.org/[email protected]/server.go:1737 +0x9e7
    google.golang.org/grpc.(*Server).serveStreams.func1.1()
    	/Users/guin/go/pkg/mod/google.golang.org/[email protected]/server.go:982 +0x8d
    created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 85
    	/Users/guin/go/pkg/mod/google.golang.org/[email protected]/server.go:980 +0x165

Output of pulumi about

Note that this repro is done on an older version of pulumi-gcp. We have created a patch to address this issue.

CLI          
Version      3.87.0
Go Version   go1.21.2
Go Compiler  gc

Plugins
NAME    VERSION
gcp     6.50.0
nodejs  unknown

Host     
OS       darwin
Version  11.7.1
Arch     x86_64

This project is written in nodejs: executable='/Users/guin/.nvm/versions/node/v20.6.0/bin/node' version='v20.6.0'

Current Stack: guinevere-pulumi-corp/typescript/dev

Found no resources associated with dev

Found no pending operations associated with dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/guinevere-pulumi-corp
User           guinevere-pulumi-corp
Organizations  guinevere-pulumi-corp, pulumi
Token type     personal

Dependencies:
NAME            VERSION
@pulumi/gcp     6.50.0
@pulumi/pulumi  3.87.0
@types/node     16.18.58

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[VenelinMartinov]

This was fixed with a patch, so not a P1.

[VenelinMartinov]

The panic is still there without the patch, we should investigate further.

[mjeffryes]

Going to keep this as a P1 until we verify if there's a bridge bug. We want to be sure other providers are not affected.

[VenelinMartinov]

Root caused to pulumi/pulumi-terraform-bridge#2282

The issue is that the Importer method is not called but the bridge calls TF Refresh without a resource ID. This in turn, leads to the wrong GCP API call - it calls the GET method on
https://bigquery.googleapis.com/bigquery/v2/projects/pulumi-development/datasets/
instead of
https://bigquery.googleapis.com/bigquery/v2/projects/pulumi-development/datasets/mydataset,

which returns the wrong format for the response, breaking the code.

Note that the GCP .get operation for the resource is still broken - it calls the wrong API and just doesn't panic anymore.

[iwahbe]

Because this is not a regression (it never worked) and because we no longer panic (with the patch), there is no reason for this to be a P1.