A complete guide to terraform count and for_each and dynamic blocks with examples

Terraform stands out as a powerful tool for managing and provisioning cloud resources. However, as your infrastructure grows more complex, you might find yourself repeating code blocks, leading to verbose and hard-to-maintain configurations.

To address this, Terraform offers a dynamic trio, count, for_each, and dynamic blocks.

In this blog post, I’ll explain these three super useful features in Terraform that’ll make your code cleaner, more flexible, and DRY.

Read about the evolution of infrastructure as a code concept and the rise of terraform.

Let’s get started

By default, when you use a resource block in Terraform, it sets up one actual infrastructure object. Similarly, when you use a module block, it brings in the contents of a child module just once.

But there are situations where you might need to handle multiple similar objects, like having a group of compute instances, without creating a separate block for each.

Terraform provides two methods for doing this: `count` and `for_each`.

The count Meta-Argument

The count is a special command in the Terraform language. It works with both modules (starting from Terraform version 0.13) and every type of resource.

When you use the count command, you provide a whole number, and Terraform then creates that exact number of instances for the specified resource or module.

Each instance represents a distinct infrastructure object, and each one is managed individually during the apply process, whether it’s for creation, updating, or removal.

Let’s take a basic example of creating multiple ec2 instances in AWS.

resource "aws_instance" "server" {
  count = 4 # create four similar EC2 instances

  ami           = "ami-a1b2c3d4"
  instance_type = "t2.micro"

  tags = {
    Name = "Server ${count.index}"
  }
}

In Terraform, count.index is a special variable that represents the current index(starting with 0) of the resource being created when using the count meta-argument.

Here are some common use cases of count meta-arguments in python

You can iterate over lists to create instances with different configurations.

variable "instance_types" {
  type    = list(string)
  default = ["t2.micro", "t2.small", "t2.medium"]
}

resource "aws_instance" "example" {
  count = length(var.instance_types)
  ami   = "ami-0c55b159cbfafe1f0"
  instance_type = var.instance_types[count.index]
}

It will create 3 VMs with different machine-type

count functions can be used in modules to create multiple instances of a module.

module "example_module" {
  source = "./modules/example"
  count  = 2
}

Conditional Resource Creation
count.index variable allows dynamic resource naming based on the instance index.

variable "create_instances" {
  type    = bool
  default = true
}

resource "aws_instance" "example" {
  count         = var.create_instances ? 3 : 0
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
  tags = {
    Name = "instance-${count.index + 1}"
  }
}

If the create_instance variable is set to true, it will create 3 VM, else no VM will be created.

count.index can be used to dynamically configure resources based on the instance index.

resource "aws_instance" "example" {
  count         = 4
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
  
  private_ip = "10.0.1.${count.index + 1}"
}

Assigning private IP addresses based on the index. Each instance gets an IP in the range 10.0.1.1 to 10.0.1.4.

The for_each Meta-Argument

Unlike the count meta-argument, which creates a fixed number of instances, for_each allows you to dynamically create resources based on a map or set of key-value pairs.

The for_each meta-argument accepts a map or a set of strings and creates an instance for each item in that map or set.

Take a look at the below example

variable "app_configs" {
  type = map(string)
  default = {
    web   = "web-app"
    db    = "database"
    cache = "cache-cluster"
  }
}

resource "aws_instance" "example" {
  for_each = var.app_configs

  ami           = "ami-98765432"
  instance_type = "t2.micro"
  tags = {
    Name = each.value
  }
}

for_each meta-argument is utilized to dynamically name resources based on the values in the map.

The instances are named “web-app,” “database,” and “cache-cluster” accordingly.

We can use for_each to create a more complex configuration.

Dynamic Scaling with Map

variable "instance_configs" {
  type = map(object({
    ami   = string
    size  = string
    count = number
  }))
  default = {
    web   = { ami = "ami-12345678", size = "t2.micro", count = 2 }
    db    = { ami = "ami-87654321", size = "t2.small", count = 1 }
    cache = { ami = "ami-56789012", size = "t2.medium", count = 3 }
  }
}

resource "aws_instance" "example" {
  for_each = var.instance_configs

  ami           = each.value.ami
  instance_type = each.value.size
  count         = each.value.count

}

We can dynamically scale resources based on different configurations given in the map variable.

Conditional Resource Creation

variable "create_apps" {
  type    = map(bool)
  default = {
    web   = true
    db    = false
    cache = true
  }
}

resource "aws_instance" "example" {
  for_each = { for k, v in var.create_apps : k => v if v }

  ami           = "ami-11112222"
  instance_type = "t2.micro"
  tags = {
    Name = each.key
  }
}

It creates instances only for the apps with a value of true in the create_apps map. We used map transformation to create a new map with true values.

Read about how to use terraform map transformation

We can use for_each with a list input, but we have to convert it into a set first using the toset() function.

resource "aws_s3_object" "base_folders" {
  for_each = toset([
    "archive",
    "inbound",
    "outbound"
  ])
  bucket = aws_s3_bucket.my_bucket.id
  acl    = "private"
  key    = "${each.value}/"

}

Note: each.key and each.value are the same for a set

Chaining `for_each` Between Resources

when you use for_each to create multiple instances of something (like VPCs), you can then use those instances directly to create another set of things (like internet gateways).

variable "vpcs" {
  type = map(object({
    cidr_block = string
  }))
}

resource "aws_vpc" "example" {
  # One VPC for each element of var.vpcs
  for_each = var.vpcs

  # each.value here is a value from var.vpcs
  cidr_block = each.value.cidr_block
}

resource "aws_internet_gateway" "example" {
  # One Internet Gateway per VPC
  for_each = aws_vpc.example

  # each.value here is a full aws_vpc object
  vpc_id = each.value.id
}

output "vpc_ids" {
  value = {
    for k, v in aws_vpc.example : k => v.id
  }

  # The VPCs aren't fully functional until their
  # internet gateways are running.
  depends_on = [aws_internet_gateway.example]
}

When for_each is set, Terraform distinguishes between the block itself and the multiple resource or module instances associated with it.

variable "bucket_names" {
  type    = map(string)
  default = {
    marketing = "marketing-assets"
    dev       = "dev-environment"
    logs      = "access-logs"
  }
}

# Create S3 Buckets
resource "aws_s3_bucket" "example" {
  for_each = var.bucket_names

  bucket = each.value
  acl    = "private"
}

# Use the Resulting Buckets in EC2 Instances
resource "aws_instance" "example" {
  for_each = aws_s3_bucket.example

  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"

  tags = {
    Name        = "instance-${each.key}"
    Bucket_Name = aws_s3_bucket.example[each.key].bucket
  }
}

The above code created AWS S3 buckets using `for_each`, then used resulting buckets in EC2 instances, tagging each instance with a generic name and the corresponding S3 bucket name.

aws_s3_bucket.example[each.key].bucket refers to the individual buckets created with aws_s3_bucket.example block

Limitations on values used in `for_each`

The keys of the map (or all values in a set of strings) used in for_each must be known values.
Sensitive values, including sensitive input/output variables or resource attributes, cannot be used as arguments for for_each

Dynamic Blocks

Dynamic Blocks in Terraform are like smart copy-paste tools. They help you create repeated parts inside a resource without writing the same thing over and over. This keeps your code clean and easy to manage.

Here’s how they work:

for_each: This is the main part. It goes through a list or map you give it.
content: This is where you put what you want to repeat.
iterator: This is optional. It’s a temporary name for each item as you go through the list.
labels: This is also optional and not used much. It’s for naming the repeated blocks.

You can use Dynamic Blocks inside other Dynamic Blocks, or use them to create optional parts of your configuration.

Here’s a simple example.

In this example, we’re creating multiple ingress rules for a security group. Instead of writing out each rule, we use a Dynamic Block to create a rule for each port in our ingress_ports variable. This makes our code shorter and easier to change later.

variable "ingress_ports" {
default = [80, 443, 8080]
}
resource "aws_security_group" "example" {
name = "example"
}

dynamic "ingress" {
for_each = var.ingress_ports
content {
from_port = ingress.value
to_port = ingress.value
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
}

Let me use one more example using Dynamic Blocks in Terraform. In this example, we’ll create an AWS EC2 instance with multiple EBS volumes attached.

variable "additional_ebs_volumes" {
default = [
{
device_name = "/dev/sdb"
volume_type = "gp2"
volume_size = 10
encrypted = true
},
{
device_name = "/dev/sdc"
volume_type = "gp3"
volume_size = 20
encrypted = false
}
]
}resource "aws_instance" "example" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"

dynamic "ebs_block_device" {
for_each = var.additional_ebs_volumes
content {
device_name = ebs_block_device.value.device_name
volume_type = ebs_block_device.value.volume_type
volume_size = ebs_block_device.value.volume_size
encrypted = ebs_block_device.value.encrypted
}
}

tags = {
Name = "ExampleInstance"
}
}

Let’s break it down.

We start with a regular aws_instance resource. This is where we’ll add our Dynamic Block.
Inside the resource, we use the dynamic keyword to start our Dynamic Block. We name it ebs_block_device because that’s the nested block type we want to create multiple times within the EC2 instance resource.
The for_each argument is set to var.additional_ebs_volumes. This variable is a list of maps, each map represents the configuration for one EBS volume.
The content block defines what goes inside each dynamically created ebs_block_device block. Here’s where we specify the details of each EBS volume.
Inside the content block, we use ebs_block_device.value to access the current item in our loop. Remember, we’re iterating over a list of maps, so ebs_block_device.value refers to the current map.
We set each attribute of the ebs_block_device block using the corresponding value from our map:

device_name: The name of the device (like /dev/sdb)
volume_type: The type of EBS volume (like gp2 or gp3)
volume_size: The size of the volume in GB
encrypted: Whether the volume should be encrypted

7. This code will create an EC2 instance and for each item in var.additional_ebs_volumes, it will:

Create an ebs_block_device block within the EC2 instance resource
Set the attributes of this block based on the values in the current map

This Dynamic Block approach allows us to easily add, remove, or modify EBS volumes by simply changing the additional_ebs_volumes variable, without needing to alter the main resource configuration. It’s a powerful way to make our Terraform code more flexible and maintainable.

When to use dynamic blocks?

For creating multiple nested blocks within a resource
When dealing with complex resources that have multiple configurable nested blocks
To make your code more flexible and maintainable

That’s it for this blog, see you on the next one.