Skip to content

Commit

Permalink
Disambiguate backend configurations during migration
Browse files Browse the repository at this point in the history
A user reported at hashicorp/terraform-provider-consul#275
that the message when migrating backend was not detailed enough to be sure what
was the previously configured backend, and what is the new one. This can make
migrating from one backend to another dangerous as there is no way to check the
configuration is correct when actually migrating to a new backend.

The message when using the consul backend (though the issue still stands with
other backends as well) was:

	Do you want to migrate all workspaces to "consul"?
	  Both the existing "consul" backend and the newly configured "consul" backend
	  support workspaces. When migrating between backends, Terraform will copy
	  all workspaces (with the same names). THIS WILL OVERWRITE any conflicting
	  states in the destination.

	  Terraform initialization doesn't currently migrate only select workspaces.
	  If you want to migrate a select number of workspaces, you must manually
	  pull and push those states.

	  If you answer "yes", Terraform will migrate all states. If you answer
	  "no", Terraform will abort.

I changed the Backend interface to add a String() method that returns an HCL
representation of the backend and used it to return more information when migrating.
The new message is:

	Do you want to copy existing state to the new backend?
	  Pre-existing state was found while migrating the previous "local" backend to the
	  newly configured "consul" backend. No existing state was found in the newly
	  configured "consul" backend.

	  The configuration for the previous "local" backend was:

	    backend "local" {
	      path          = ""
	      workspace_dir = ""
	    }

	  The configuration for the new "consul" backend is:

	    backend "consul" {
	      access_token = ""
	      address      = ""
	      ca_file      = ""
	      cert_file    = ""
	      datacenter   = ""
	      gzip         = false
	      http_auth    = ""
	      key_file     = ""
	      lock         = true
	      path         = "full/path"
	      scheme       = ""
	    }

	  Do you want to copy this state to the new "consul"
	  backend? Enter "yes" to copy and "no" to start with an empty state.

For the local and remote backends the implementation is straightforward, for all
the others the representation is generated from the backend's Schema and its
configuration.
  • Loading branch information
remilapeyre committed Sep 13, 2021
1 parent 0a31fa0 commit b1271dd
Show file tree
Hide file tree
Showing 23 changed files with 129 additions and 36 deletions.
4 changes: 4 additions & 0 deletions backend/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,10 @@ type Backend interface {
// States returns a list of the names of all of the workspaces that exist
// in this backend.
Workspaces() ([]string, error)

// String returns a valid string representation of the backend as it would
// appear in the Terraform configuration
String() string
}

// Enhanced implements additional behavior on top of a normal backend.
Expand Down
12 changes: 12 additions & 0 deletions backend/local/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ import (
"os"
"path/filepath"
"sort"
"strings"
"sync"

"github.com/hashicorp/hcl/v2/hclwrite"
"github.com/hashicorp/terraform/backend"
"github.com/hashicorp/terraform/command/clistate"
"github.com/hashicorp/terraform/configs/configschema"
Expand Down Expand Up @@ -344,6 +346,16 @@ func (b *Local) Operation(ctx context.Context, op *backend.Operation) (*backend.
return runningOp, nil
}

func (b *Local) String() string {
f := hclwrite.NewFile()

body := f.Body().AppendNewBlock("backend", []string{"local"}).Body()
body.SetAttributeValue("path", cty.StringVal(b.StatePath))
body.SetAttributeValue("workspace_dir", cty.StringVal(b.StateWorkspaceDir))

return strings.TrimSpace(string(f.Bytes()))
}

// opWait waits for the operation to complete, and a stop signal or a
// cancelation signal.
func (b *Local) opWait(
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/artifactory/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (

func New() backend.Backend {
s := &schema.Backend{
Type: "artifactory",
Schema: map[string]*schema.Schema{
"username": &schema.Schema{
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/azure/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
// New creates a new backend for Azure remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "azure",
Schema: map[string]*schema.Schema{
"storage_account_name": {
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/consul/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
// New creates a new backend for Consul remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "consul",
Schema: map[string]*schema.Schema{
"path": &schema.Schema{
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/cos/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ type Backend struct {
// New creates a new backend for TencentCloud cos remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "cos",
Schema: map[string]*schema.Schema{
"secret_id": {
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/etcdv2/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (

func New() backend.Backend {
s := &schema.Backend{
Type: "etcd",
Schema: map[string]*schema.Schema{
"path": &schema.Schema{
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/etcdv3/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ const (

func New() backend.Backend {
s := &schema.Backend{
Type: "etcdv3",
Schema: map[string]*schema.Schema{
endpointsKey: &schema.Schema{
Type: schema.TypeList,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/gcs/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ type Backend struct {
func New() backend.Backend {
b := &Backend{}
b.Backend = &schema.Backend{
Type: "gcs",
ConfigureFunc: b.configure,
Schema: map[string]*schema.Schema{
"bucket": {
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/http/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (

func New() backend.Backend {
s := &schema.Backend{
Type: "http",
Schema: map[string]*schema.Schema{
"address": &schema.Schema{
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/inmem/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ func Reset() {
func New() backend.Backend {
// Set the schema
s := &schema.Backend{
Type: "inmem",
Schema: map[string]*schema.Schema{
"lock_id": &schema.Schema{
Type: schema.TypeString,
Expand Down
3 changes: 2 additions & 1 deletion backend/remote-state/kubernetes/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import (
const (
noConfigError = `
[Kubernetes backend] Neither service_account nor load_config_file were set to true,
[Kubernetes backend] Neither service_account nor load_config_file were set to true,
this could cause issues connecting to your Kubernetes cluster.
`
)
Expand All @@ -42,6 +42,7 @@ var (
// New creates a new backend for kubernetes remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "kubernetes",
Schema: map[string]*schema.Schema{
"secret_suffix": {
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/manta/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (

func New() backend.Backend {
s := &schema.Backend{
Type: "manta",
Schema: map[string]*schema.Schema{
"account": {
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/oss/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import (
// New creates a new backend for OSS remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "oss",
Schema: map[string]*schema.Schema{
"access_key": &schema.Schema{
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/pg/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ const (
// New creates a new backend for Postgres remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "pg",
Schema: map[string]*schema.Schema{
"conn_str": {
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/s3/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
// New creates a new backend for S3 remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "s3",
Schema: map[string]*schema.Schema{
"bucket": {
Type: schema.TypeString,
Expand Down
1 change: 1 addition & 0 deletions backend/remote-state/swift/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ type Config struct {
// New creates a new backend for Swift remote state.
func New() backend.Backend {
s := &schema.Backend{
Type: "swift",
Schema: map[string]*schema.Schema{
"auth_url": {
Type: schema.TypeString,
Expand Down
21 changes: 21 additions & 0 deletions backend/remote/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import (

tfe "github.com/hashicorp/go-tfe"
version "github.com/hashicorp/go-version"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/hashicorp/hcl/v2/hclwrite"
svchost "github.com/hashicorp/terraform-svchost"
"github.com/hashicorp/terraform-svchost/disco"
"github.com/hashicorp/terraform/backend"
Expand Down Expand Up @@ -792,6 +794,25 @@ func (b *Remote) Operation(ctx context.Context, op *backend.Operation) (*backend
return runningOp, nil
}

func (b *Remote) String() string {
f := hclwrite.NewFile()

body := f.Body().AppendNewBlock("backend", []string{"remote"}).Body()
body.SetAttributeValue("hostname", cty.StringVal(b.hostname))
body.SetAttributeValue("organization", cty.StringVal(b.organization))
body.SetAttributeRaw("token", hclwrite.Tokens{&hclwrite.Token{
Type: hclsyntax.TokenIdent,
Bytes: []byte("(sensitive)"),
}})
body.AppendNewline()

body = body.AppendNewBlock("workspaces", []string{}).Body()
body.SetAttributeValue("name", cty.StringVal(b.workspace))
body.SetAttributeValue("prefix", cty.StringVal(b.prefix))

return strings.TrimSpace(string(f.Bytes()))
}

func (b *Remote) cancel(cancelCtx context.Context, op *backend.Operation, r *tfe.Run) error {
if r.Actions.IsCancelable {
// Only ask if the remote operation should be canceled
Expand Down
4 changes: 4 additions & 0 deletions builtin/providers/terraform/data_source_state_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -369,3 +369,7 @@ func (b backendFailsConfigure) DeleteWorkspace(name string) error {
func (b backendFailsConfigure) Workspaces() ([]string, error) {
return nil, fmt.Errorf("Workspaces not implemented")
}

func (b backendFailsConfigure) String() string {
return ""
}
31 changes: 26 additions & 5 deletions command/meta_backend_migrate.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (

"github.com/hashicorp/terraform/backend"
"github.com/hashicorp/terraform/command/clistate"
"github.com/hashicorp/terraform/internal/logging"
"github.com/hashicorp/terraform/states"
"github.com/hashicorp/terraform/states/statemgr"
"github.com/hashicorp/terraform/terraform"
Expand Down Expand Up @@ -412,7 +413,8 @@ func (m *Meta) backendMigrateEmptyConfirm(one, two statemgr.Full, opts *backendM
Query: "Do you want to copy existing state to the new backend?",
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateEmpty),
opts.OneType, opts.TwoType),
opts.OneType, opts.TwoType,
logging.Indent(opts.One.String()), logging.Indent(opts.Two.String())),
}

return m.confirm(inputOpts)
Expand Down Expand Up @@ -453,7 +455,8 @@ func (m *Meta) backendMigrateNonEmptyConfirm(
Query: "Do you want to copy existing state to the new backend?",
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateNonEmpty),
opts.OneType, opts.TwoType, onePath, twoPath),
opts.OneType, opts.TwoType, onePath, twoPath,
logging.Indent(opts.One.String()), logging.Indent(opts.Two.String())),
}

// Confirm with the user that the copy should occur
Expand Down Expand Up @@ -496,7 +499,7 @@ This will attempt to copy (with permission) all workspaces again.
`

const errBackendStateCopy = `
Error copying state from the previous %q backend to the newly configured
Error copying state from the previous %q backend to the newly configured
%q backend:
%s
Expand All @@ -507,7 +510,17 @@ the error above and try again.
const inputBackendMigrateEmpty = `
Pre-existing state was found while migrating the previous %q backend to the
newly configured %q backend. No existing state was found in the newly
configured %[2]q backend. Do you want to copy this state to the new %[2]q
configured %[2]q backend.
The configuration for the previous %[1]q backend was:
%[3]s
The configuration for the new %[2]q backend is:
%[4]s
Do you want to copy this state to the new %[2]q
backend? Enter "yes" to copy and "no" to start with an empty state.
`

Expand All @@ -520,6 +533,14 @@ removed after responding to this query.
Previous (type %[1]q): %[3]s
New (type %[2]q): %[4]s
The configuration for the previous %[1]q backend was:
%[5]s
The configuration for the new %[2]q backend is:
%[6]s
Do you want to overwrite the state in the new backend with the previous state?
Enter "yes" to copy and "no" to start with the existing state in the newly
configured %[2]q backend.
Expand Down Expand Up @@ -550,7 +571,7 @@ If you answer "yes", Terraform will migrate all states. If you answer

const inputBackendNewWorkspaceName = `
Please provide a new workspace name (e.g. dev, test) that will be used
to migrate the existing default workspace.
to migrate the existing default workspace.
`

const inputBackendSelectWorkspace = `
Expand Down
Loading

0 comments on commit b1271dd

Please sign in to comment.