Goal of this terraform code is to stand up working environments, from scratch, following best practices. All the environments are almost identical and setup around GKE as a core for application deployments. All the code is designed to work with terraform cloud. Terraform code is structured to be deployed in various workspaces. Please see this link on how to organize the code for terraform cloud. The following list shows what resources are deployed in which folder:
- infra-host-project
- Environment Directory
- Project that hosts a VPC
- Shared VPC
- Cloud NAT
- infra-service-project
- Service Project that uses shared VPC created above.
- Service accounts for various current and future application and resources
- security
- IAM bindings for service accounts created above and for admin, secops, and viewer groups
- Firewall rules
- application
- private GKE cluster
- gke bastion host that can be used to access the GKE cluster
Order of deployment is infra-host-project
-> infra-service-project
-> security
-> application
Following are the prerequisites necessary for successful deployment of resources.
Before you start deploying this code i would strongly recommend that you clone this repo to your own github account.
- You will need to create a terraform cloud account at https://app.terraform.io
- Once you have your account, create a token and store it in ~/.terraformrc. Please see: https://www.terraform.io/docs/cloud/free/index.html#configure-access-for-the-terraform-cli
- In Terraform UI create an organization. See: https://www.terraform.io/docs/cloud/free/index.html#create-an-organization
- Once you have your organization, connect your github account with terraform cloud. Follow these instructions: https://www.terraform.io/docs/cloud/vcs/github.html
- Now that you have your organization and github oath_token_id export set the environment variables like this:
export TF_CLOUD_ORG=<Your Cloud Organization Name> export TF_CLOUD_OAUTH_TOKEN_ID=<OAuth Token ID>
- Create a project called bootstrap-gcp-lz
- Enable the following APIs if they are not enabled:
- iam.googleapis.com
- container.googleapis.com
- cloudresourcemanager.googleapis.com
- Create a service account called
bootstrap
- Assign the following roles to
bootstrap
service account:Billing Account User
on the org levelCompute Shared VPC Admin
on the org levelFolder Creator
on the org levelOrganization Viewer
on the org levelProject Creator
on the org levelOwner
on the project level
- Create a key-pair credentials for the
bootstrap
service account. (It will be downloaded to your computer) - Place the credentials file in the root of this repo and rename it to
credentials.json
You must complete all the steps in Prerequistes section. It is very important that you set these:
export TF_CLOUD_ORG=<Your Cloud Organization Name>
export TF_CLOUD_OAUTH_TOKEN_ID=<OAuth Token ID>
cd workspaces
./create-workspaces.py --org $TF_CLOUD_ORG --oauth_token_id $TF_CLOUD_OAUTH_TOKEN_ID
- This will create workspaces per environment. For example the there will be three workspaces for infra-host-project: infra-host-project-dev, infra-host-project-stg, infra-host-project-prd
- workspace details are in workspaces.yaml
./pushvars.py --org $TF_CLOUD_ORG *.hcl
- This will populate variables for each workspace created in the previous step
We treat GCP organization ID and Billing Account Id as sensitive information, therefore those values should not be committed to a git repo.
- Go to your terraform cloud UI.
- Manually add the following variables in infra-host-project-dev, infra-host-project-stg, infra-host-project-prd,
infra-service-project-dev, infra-service-project-stg, infra-service-project-prd:
- org_id =
your gcp organization ID
- billing_account_id =
your organization billing account id
- org_id =
Dev environment is meant for terraform code development. Workspaces for dev environment are not connected to VCS though they can be if you want. (modify workspaces/workspaces.yaml if you want this feature enabled on dev)
- Deploy infra-host-project
cd infra-host-project
cp backend.hcl.example backend.hcl
- Modify
backend.hcl
with proper information
organization = "<my-terraform-cloud-organization>" workspaces { name = "infra-host-project-dev" }
terraform init -backend-config backend.hcl
terraform plan
terraform apply
- Deploy infra-service-project
cd infra-host-project
cp backend.hcl.example backend.hcl
- Modify
backend.hcl
with proper information
organization = "<my-terraform-cloud-organization>" workspaces { name = "infra-service-project-dev" }
terraform init -backend-config backend.hcl
terraform plan
terraform apply
- Deploy security
cd security
cp backend.hcl.example backend.hcl
- Modify
backend.hcl
with proper information
organization = "<my-terraform-cloud-organization>" workspaces { name = "security-dev" }
terraform init -backend-config backend.hcl
terraform plan
terraform apply
- Deploy application
cd application
cp backend.hcl.example backend.hcl
- Modify
backend.hcl
with proper information
organization = "<my-terraform-cloud-organization>" workspaces { name = "application-us-central1-dev" }
terraform init -backend-config backend.hcl
terraform plan
terraform apply
These environments can be deployed via terraform UI.
- Go to your terraform cloud UI
- Go to your organization
- Queue up the following workspaces in this particular order:
- infra-host-project-stg
- infra-service-project-stg
- security-stg
- application-stg
- Repeat the same for production environment
Staging and Production workspaces are connected to VCS repo and will be triggered if there is any changes in their respective directories. These environments are designed in this manner so that there are no manual deployment to staging and production. These environments should always be deployed/redeployed via UI on PRs or other change requests.