Terraform Lookup

Infrastructure as Code (IaC) is one of the best ways to automate and scale infrastructure to keep up with the rapid pace of modern software development. With IaC, engineers can codify infrastructure and directly integrate changes into CI/CD pipelines. 

Terraform is one of the most popular IaC tools for modern DevOps teams. However, it takes knowledge and practice to get the most out of Terraform. In this article, we’ll help you improve your IaC skills by taking a deep dive into Terraform lookup functions and maps. Lookup functions and maps provide you with a great way to streamline and optimize your Terraform code. In addition to exploring concepts such as the lookup function, maps, and elements, we’ll provide practical examples to help you get hands-on with Terraform.

What are Terraform Functions?

Terraform utilizes the domain-specific Hashicorp Configuration Language (HCL) as the primary method for writing infrastructure configurations. HCL can be used to create infrastructure configurations across hundreds of providers. However, it does not support user-defined functions. As a result, HCL functions are limited to built-in HCL functions. You can call Terraform functions from within expressions to enable specific functionalities like transforming or combining values. Functions range from simple numeric or string functions to more complex functionalities such as Hash and crypto and type conversions.

What is a Terraform Map?

It helps to understand how Terraform defines the concept of a map before you learn about the lookup function. Maps in Terraform are a type of input variable that store multiple key-value pairs. While a single variable stores  a single value, maps can store multiple values as key-value pairs. Terraform maps typically provide a group of values such as server images, ssh keys, etc., and allow users to select a specific value depending on their use case.

What is the Terraform Lookup Function?

The Terraform lookup function is one that falls under the built-in functions category. It retrieves a single value of an element in a map.

This function takes the map name, key, and a default value as its arguments and looks for the key in the specified map. It will return the value of the key if a matching key is found, or otherwise, it will return the default value.

Automated FinOps for Hybrid Cloud Management

Learn More

Customizable guardrails to embed FinOps policies into new workload provisioning

Self-service catalogs to provision compliant, cost-efficient resources in private and public clouds

Accountable FinOps workflows with task assignment, chargeback reports, and scorecards

Lookup Function Syntax

lookup(<map name>, <key>, <default value>)

The default value is an optional argument. However, it is best practice to add the default value to eliminate function call errors that occur due to lookup failures.

Why use a lookup Function?

The lookup function can act as a search function for maps. It enables users to parse through any map and extract a specific value. For example, suppose your infrastructure configuration consists of multiple compute instances with different AMIs. You can create a map to store all the IDs of your different AMIs to maintain them as a single object.

However, you will need a way to look up this map to retrieve the exact value you need. This is where the lookup function comes into play by allowing you to retrieve a specified AMI. Since the lookup function can be configured with a default value, it will not return an error even if the map does not have the specified key. In that case, the defined default value will be used in the configuration.

Terrafrom Lookup vs Element Function

The lookup function is not the only way to retrieve a specific value from a group of values. You can also use the element function. However, the difference lies in the targeted object type of these functions. The element function is used to iterate and extract values from a list, while lookup is targeted at maps.

A list is an ordered sequence of strings indexed using integers. Both lookup and element functions can be used to store multiple values and manage them as single objects. However, lookup allows users to lookup values within maps using a specific key, while element requires users to specify the exact index to return the corresponding value from a list.

Another difference is that lookup allows users to configure a default value to be used when the specified key is not available. An element does not. The element function will return an error if the specified index is invalid.

An element with a list is ideal if you need to specify an exact value in your configuration without any substitutes. However, you must know the exact indexes of the items in the list. Lookup with maps is a more flexible approach as it allows users to look up values using a specific key even without knowing the exact index. Additionally, it will allow continuing the configuration without failure using the default value even when the key is unavailable.

Basic Terraform Lookup Function Usage

Now that you understand the functionality of the lookup function, let’s look at some examples of retrieving a value from a map. For these examples we used Windows 10 with Terraform v 1.1.4 and Amazon Web Services as the target cloud environment with Terraform AWS provider v 3.74 (hashicorp/AWS). We will use the terraform plan command to obtain the required output.

Example 1 – Terraform Lookup with a simple map

Assume there is a map of AMI IDs, and you want to use the specified value in your Terraform configuration. Here, we will be using the output command to print the results of the lookup function for simplicity.

# AMI Collection Map
variable "ami_collection" {
    type = map(string)


    default = {
      "ubuntu" = "ami-00ae935ce6c2aa534"
      "amazon_linux" = "ami-00ae935ce6c2aa534"
      "rhel_sql" = "ami-0fd0947c3f88732f8"
      "windows_server_2019" = "ami-00ae935ce6c2aa534"
      "windows_server_2022" = "ami-0f96fbe09adbebdc9"
    }
}


# Selecting a available key (ubuntu)
output "select_ami" {
    value = lookup(var.ami_collection, "ubuntu", "ami-0de899d345371c9aa")
}


# Selecting an unavailable key
output "select_ami_default" {
    value = lookup(var.ami_collection, "ubuntu_server", "ami-0de899d345371c9aa")
}

Output

terraform plan

Changes to Outputs:
  + select_ami         = "ami-00ae935ce6c2aa534"
  + select_ami_default = "ami-0de899d345371c9aa"

In this example, we have a simple map called ami_collection with five key-value pairs consisting of AMI IDs. We have defined “ubuntu” as the key in the select_ami output, and it returns the value for that key as it matches with a  specified key in the map. We have specified the key as “ubuntu_server” in the second output, select_ami_default, and the lookup function will return the given default value as there is no matching key in the map.

Example 2 – Terraform Lookup with an empty map

In the example below, we defined an empty map called “ami_collection_empty”. There, we are querying for a key called “ubuntu” using the lookup function. However, there is no matching key as the specified map is empty, and it will return the default value.

# Empty Map
variable "ami_collection_empty" {
    type = map(string)


    default = {}
}


output "select_ami_empty" {
    value = lookup(var.ami_collection_empty, "ubuntu", "ami-0de899d345371c9aa")
}

Output

terraform plan

Changes to Outputs:
  + select_ami_empty = "ami-0de899d345371c9aa"

Example 3 – Terraform Lookup with a nested map

The following example shows a nested map that can be queried. However, it is impossible to query a specific key within a nested map.

# Nested Map
variable "ami_collection_nested" {
    type = map


    default = {
        "linux" = {
            "ubuntu" = "ami-00ae935ce6c2aa534"
            "amazon_linux" = "ami-00ae935ce6c2aa534"
            "rhel_sql" = "ami-0fd0947c3f88732f8"
        }
        "windows" = {
            "windows_server_2019" = "ami-00ae935ce6c2aa534"
            "windows_server_2022" = "ami-0f96fbe09adbebdc1"
        }
    }
}


# Default Map
variable "other" {
    type = map


    default = {
        "amazon_linux_secondary" = "ami-0de899d345371c9aa"
    }
}


output "select_ami_nested" {
    value = lookup(var.ami_collection_nested, "linux", var.other)
}

Output

terraform plan

Changes to Outputs:
  + select_ami_nested = {
      + "amazon_linux" = "ami-00ae935ce6c2aa534"
      + "rhel_sql"     = "ami-0fd0947c3f88732f8"
      + "ubuntu"       = "ami-00ae935ce6c2aa534"
    }

In this example, we are querying a nested map with the key “linux” and the complete map is retired as the result as it matches a map within the nested map ami_collection_nested. Remember we need to specify a default value as a map, or else it will result in an invalid function argument error.

Dynamic Operations with Terraform Lookup Function

Here, we have defined two EC2 instances with their AMI IDs obtained through the lookup function from a map named production_ami_collection.

# Security Group for Web Servers
resource "aws_security_group" "test_web_server_sg" {
  name        = "web-server-sg"
  description = "Web Server Access"
  vpc_id      = "vpc-a3xxxxx"


  ingress {
    from_port        = 443
    to_port          = 443
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }


  ingress {
    from_port        = 22
    to_port          = 22
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }


  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }


  tags = {
    Name = "web-server-sg"
    Env  = "production"
  }
}


# AMI Collection Map
variable "ami_collection" {
  type = map(string)


  default = {
    "ubuntu"              = "ami-00ae935ce6c2aa534"
    "amazon_linux"        = "ami-00ae935ce6c2aa534"
    "rhel_sql"            = "ami-0fd0947c3f88732f8"
    "windows_server_2019" = "ami-00ae935ce6c2aa534"
    "windows_server_2022" = "ami-0f96fbe09adbebdc1"
  }
}


# Create a Ubuntu based Instance
resource "aws_instance" "web_server_project_red" {
  # Lookup the Ubuntu AMI ID
  ami                         = lookup(var.ami_collection, "ubuntu", "ami-0de899d345371c9aa")
  instance_type               = "t3a.nano"
  availability_zone           = "eu-central-1a"
  subnet_id                   = "subnet-cf5faf94"
  associate_public_ip_address = true
  vpc_security_group_ids      = [aws_security_group.test_web_server_sg.id]
  key_name                    = "frankfurt-elastic-agent-key"
  disable_api_termination     = true
  monitoring                  = true


  depends_on = [
    aws_security_group.test_web_server_sg
  ]


  credit_specification {
    cpu_credits = "standard"
  }


  root_block_device {
    delete_on_termination = true
    volume_size           = 30
  }


  tags = {
    Name = "[web-server]project-red"
    Env  = "production"
  }
}


# Create a Windows based Instance
resource "aws_instance" "web_server_project_green" {
  # Lookup the Windows Server 2022 AMI ID
  ami                         = lookup(var.ami_collection, "windows_server_2022", "ami-0de899d345371c9aa")
  instance_type               = "t3a.nano"
  availability_zone           = "eu-central-1a"
  subnet_id                   = "subnet-cf5faf94"
  associate_public_ip_address = true
  vpc_security_group_ids      = [aws_security_group.test_web_server_sg.id]
  key_name                    = "frankfurt-elastic-agent-key"
  disable_api_termination     = true
  monitoring                  = true


  depends_on = [
    aws_security_group.test_web_server_sg
  ]


  credit_specification {
    cpu_credits = "standard"
  }


  root_block_device {
    delete_on_termination = true
    volume_size           = 50
  }


  tags = {
    Name = "[web-server]project-green"
    Env  = "production"
  }
}

As one of the most versatile objects available in HCL, maps allow users to store simple data sets like AMI IDs to entire VPC configurations including subnet, route table, NACL, security group, etc. and manage it as a single object. You can then query the data with the look function, eliminating the need to manage individual variables and reducing the chances for misconfigurations.

Terraform Lookup Best Practices

• While the default value is optional, it’s highly recommended to use it to avoid function call errors due to lookup failures. It is especially important when dealing with larger maps and can also be helpful in troubleshooting.
• Do not query nested maps expecting to obtain values from individual keys. Instead, it will return a complete map object.
• Do not overuse lookup within your configuration, as it can lead to less readable code.
• Ensure that the default value matches up with the map type. For example, if the map is a string type, ensure that the default value is also a string. Otherwise, it will lead to type errors.
• When defining the key value in the lookup function, always ensure it is correctly spelled as keys are case-sensitive.

Conclusion

Terraform lookup is used to easily obtain map values in HCL. It allows users to efficiently use maps within their configurations without having to iterate through each item.

Even if the matching key is unavailable, Terraform can function by supplementing the default value without breaking the configuration as lookup acts as a search function with a default value. Functions such as lookup help users create more concise and clear Terraform configurations and are an excellent tool in any DevOps engineer’s IaC toolbox.