Terraform Tags

Tags are the foundational element of your cloud governance model. Without tags, your cloud environment is the equivalent of the Wild West – your operations teams will be troubleshooting from the hip and lawlessness will reign.

This article will show you how to make Terraform tags the new Sheriff-in-town and help you avoid cost blow-outs, prolonged maintenance activities and account sprawl.

Additionally, using Terraform tags will help you harness the true power of the cloud by enabling automation, and make your cloud security model ironclad. For your convenience, please find an executive summary of this article below.

Executive summary

Tagging Use-Cases
Cost allocation tags	Get transparency on who is spending what
Project organization	Make your operations teams more efficient
Automation	Unleash the true power of cloud
Access control	Lock down resources
Environment Definition	Create blast radius control between environments

Infrastructure as Code

To have a mature cloud environment you must use Infrastructure as Code (IaC). 

The idea is to treat your infrastructure the same way you treat your software – define it with repeatable scripts, config files and templates, then check it into source control. This provides the benefits of both DevOps and elements of the Software Development Lifecycle.

Infrastructure as code enables peer review, consistency, and a rapid release cycle. All major cloud providers support IaC, including AWS, Azure, and GCP.

Tagging Basics

What are tags?

Tags are simply key-value pairs, associated with cloud resources, that contain descriptive metadata.

Keys can be anything you choose, as long as they don’t conflict with any keys that your cloud provider may have reserved. The values should describe the resources they are associated with.

Why do we tag resources?

Imagine a Christmas tree with a heap of unlabelled presents underneath. While it might be fun finding out who the presents are for, the same isn’t quite true for resources (or presents) in the cloud – especially when a production system is down or deadlines are tight.

Tagging resources helps to categorize, automate and secure your cloud resources and reduces the time required for maintenance, troubleshooting and development.

Tagging Use-Cases

Project Organization

Deploying resources into cloud platforms can be so easy that basic operational hygiene becomes overlooked.

You should approach tagging your cloud resources in the same way that you approach Test Driven Development. When defining your cloud resources using Terraform, write your tag blocks first and make sure that you answer these questions in the tag metadata:

• What is this resource’s name?
• What workload does this resource belong to?
• What cost center does this resource belong to?
• Can this resource be controlled by automation?
• Should access to this resource be restricted?

By having tags defined that clearly answer these questions, your operational staff won’t waste brain cycles trying to figure out what resources do, who they belong to, or whether they can be decommissioned.

Cost Allocation Tags

IT departments have always been a huge cost center for organizations, which means that when you look at the balance sheet, it appears to cost money without generating any back. This misinterpretation occurs for several reasons.

One of them is that there often isn’t an easy way to break down the costs of running an IT department. The IT department’s resources support other parts of the organization, but these costs are bundled together as “IT Costs”.

By tagging all cloud resources with a cost center tag, you can group resources and accurately apportion the cost to other areas of the organization. Doing so can help change the narrative of the IT department being a “cost center” when in reality, it is a key enabler for the core business.

Data compliance

Regulatory standards like GDPR and voluntary security standards like SOC 2 compel engineers to store data with strict considerations for privacy. Terraform tags help in two ways.

• Categorizing and documenting systems that hold sensitive data.
• Facilitating the deployment of special protections and security patches to sensitive systems.

By neatly documenting your infrastructure using code and tagging machines that have access to sensitive data, the price of SOC 2 compliance auditing can plummet.

Automation

Unleash the true power of the cloud by automating absolutely everything.

By defining automation targets using Terraform tags, you decouple any changes in execution scope from the automation itself. As a result,  you can change an individual resource without changing an entire automation policy (or script) to make the same changes.

Some common automation use-cases:

• regular backup jobs on important data: You can apply a backup policy that only targets resources with specific tags. For example, only backing up resources that have the “Production” tag.
• stopping idle resources to optimize cloud spend: You can target idle resources to be stopped based on their tag. For example, you may decide that anything not tagged with “Production” should be turned off over the weekend.
• security patch targeting: You can group together vulnerable resources to be patched with a tag like Patch Group, then deploy the patch to only these specific resources. Check out Amazon’s patch groups docs to learn more about patch groups.

Access control

By using AWS IAM conditions in policies, it’s possible to restrict users, groups, and roles from accessing resources that have specific tags.

This is a simple and effective way to ensure that access to production resources is limited.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:StopInstances"
            ],
            "Resource": "arn:aws:ec2:*:*:instance/*",
            "Condition": {
                "StringNotEquals": {"aws:ResourceTag/Environment": "Production"}
            }
        }
    ]
}

The above policy will only allow EC2 instances to be stopped if they don’t have the Environment tag “Production”.

Environment definition

Blast-radius control between your environments is critical in decreasing the chances of an unplanned outage.

You can use tags to define which environment your cloud resources belong to and, as mentioned above, apply both automation and access control policies based on them.

While you should use tags to identify which environment your resources belong to, there is also a strong argument to use more explicit controls. Using separate AWS accounts for each environment, for example, can be a much more effective blast-radius control than simple tagging.

Tagging How-to

Define tags with Terraform

Tagging resources using Terraform is very simple – add a tags block to your resource with a tag name and value.

The following example will create an S3 bucket with a Name tag of “My bucket” and an Environment tag of “Development”.

resource "aws_s3_bucket" "test-bucket" {
  bucket = "my-tf-test-bucket"

  tags = {
    Name        = "My bucket"
    Environment = "Development"
  }
}

Default tags

You may find yourself in the situation where you’re tagging all of your resources with the same tags – this can happen if you define an individual AWS provider for each environment account.

Instead of individually tagging your resources with the same values, you can declare default tags at the provider level, which will then be applied to all resources deployed by that provider.

provider "aws" {
  alias = "development"
  profile = "development_account"
  default_tags {
    tags = {
      Environment = "Development"
    }
  }
}

provider "aws" {
  alias = "production"
  profile = "production_account"
  default_tags {
    tags = {
      Environment = "Production"
    }
  }
}

You can override any of the provider-level tags at a resource level if you wish.

resource "aws_s3_bucket" "test-bucket" {
  provider = "aws.development"
  bucket = "my-tf-test-bucket"

  tags = {
    Environment = "Staging"
  }
}

Alternatively, you can set non-default tags in variables and use “merge” instead of just overriding them. Using an example from Terraform’s resource tagging documentation.

# Terraform 0.12 and later syntax
variable "additional_tags" {
  default     = {}
  description = "Additional resource tags"
  type        = map(string)
}

resource "aws_vpc" "example" {
  # ... other configuration ...

  # This configuration combines some "default" tags with optionally provided additional tags
  tags = merge(
    var.additional_tags,
    {
      Name = "MyVPC"
    },
  )
}

You can also have multiple sets of default tags, and choose which set to use at the resource level, using the provider alias.

provider "aws" {
  alias = "development-owner-a"
  default_tags {
    tags = {
      Environment = "Development"
      Owner = "PersonA"
    }
  }
}

provider "aws" {
  alias = "development-owner-b"
  default_tags {
    tags = {
      Environment = "Development"
      Owner = "PersonB"
    }
  }
}

resource "aws_s3_bucket" "test-bucket-1" {
  provider = "aws.development-owner-a"
  bucket = "my-tf-test-bucket-1"
}

resource "aws_s3_bucket" "test-bucket-2" {
  provider = "aws.development-owner-b"
  bucket = "my-tf-test-bucket-2"
}

Ignoring changes to tags

When you deploy tagged resources with Terraform, the tag values go into the Terraform state file. This is checked each time a plan or apply is run.  Sometimes, there will be tags on resources controlled by Terraform that are changed by external services and applications. For example, there may be a malware scanning tool that updates tags on your S3 buckets after each scan. If the scanning tool updates the tag, then your resource has drifted from the state it was in when Terraform last ran an apply.

In general, you should avoid Terraform controlled resources being modified, but tags are a special case. You can instruct Terraform to ignore certain tags by using the lifecycle block.

resource "aws_s3_bucket" "test-bucket" {

  bucket = "my-tf-test-bucket"

  tags = {
    Environment = "Staging"
    LastScanned = "Never"
  }

  lifecycle = {
    ignore_changes = tags.LastScanned
  }
}

You can also globally ignore tags at the provider level, if you wish.

provider "aws" {
  default_tags {
    tags = {
      Environment = "Development"
    }
  }
  ignore_tags {
    keys = ["LastScanned"]
  }
}

In addition to ignoring tags by prefix instead of provider, you can use the key_prefixes option like so.

provider "aws" {
  default_tags {
    tags = {
      Environment = "Development"
    }
  }

  ignore_tags {
    key_prefixes = ["kubernetes.io/"]
  }
}

Tagging Shared Resources on AWS

Sharing resources across AWS accounts using Resource Access Manager does not maintain the tags defined from the source account within the target account.

To set tags on these resources using IaC, you can define individual tag objects that refer to the IDs of the shared resources.

resource "aws_ec2_tag" "my_instance" {
  resource_id = [EC2_ID]
  key         = "Owner"
  value       = "PersonB"
}

When doing this, you must ensure that any individually defined tag objects do not conflict with tags defined in other resource blocks. A conflict may cause a perpetual difference where each definition will continuously attempt to update the other.

Wrapping up

Appropriately tagging your cloud resources has many benefits, and Terraform provides flexible mechanisms that make the implementation of this easy.

Take the time to define mandatory tags for your cloud resources and write your tag blocks first. Furthermore, always consider implementing automated tagging enforcement to ensure that your cloud environment doesn’t end up like the Wild West. If you liked this article and would like to learn more, why not check out some of our other helpful Terraform topics?