When I first started writing Terraform, I was terrible at it. I found myself drowning in a sea of repetitive blocks, each stuffed with hardcoded values. A simple change required me to update my code at multiple places, and the code did not have provision to scale. Maintaining that crappy code became a nightmare.

One of my seniors introduced me to Terraform transformation tricks and showed me how to use Terraform map transformations (using for loop to manipulate the config) to write a Terraform code that was readable, scalable, and looked good. It was flexible enough to accommodate multiple changes by updating the configuration in one place.

This wasn’t just a coding technique; it was a paradigm shift in how I approached infrastructure as code.

How To Use For Loop in Terraform For Map Transformation

Map transformation using for loops enhances code readability, maintainability, and flexibility in your Terraform configurations.

Let me show you what I mean by using a few examples.

Worth reading: These Terraform Functions Are The Life Saver

Example1: Creating multiple Lambda functions with S3 trigger

We will use a local variable lambda_config to define the Lambda Functions configuration.# config.tf

locals {
lambda_config = {
“image_processor” = {
handler = “index.handler”
runtime = “nodejs14.x”
memory_size = 256
timeout = 60
environment = {
PROCESS_TYPE = “image”
}
s3_trigger = {
bucket = “incoming-images-bucket”
events = [“s3:ObjectCreated:*”]
filter_prefix = “uploads/”
filter_suffix = “.jpg”
}
},
“document_converter” = {
handler = “converter.main”
runtime = “python3.8”
memory_size = 512
timeout = 120
environment = {
CONVERT_TO = “pdf”
}
s3_trigger = {
bucket = “incoming-documents-bucket”
events = [“s3:ObjectCreated:*”]
filter_prefix = “raw/”
filter_suffix = “.docx”
}
},
“video_transcoder” = {
handler = “transcoder.handle”
runtime = “nodejs14.x”
memory_size = 1024
timeout = 300
environment = {
OUTPUT_FORMAT = “mp4”
}
s3_trigger = {
bucket = “incoming-videos-bucket”
events = [“s3:ObjectCreated:*”]
filter_prefix = “new/”
filter_suffix = “.avi”
}
}
}
}

I will manipulate the config to create multiple local variables to use in my Terraform resource blocks.locals {

# Derive lambda names
# This creates a map of Lambda function names, appending the workspace name for uniqueness
lambda_names = {
for k, v in local.lambda_config : k => “${k}-function-${terraform.workspace}”
}

# Derive S3 bucket triggers
# This extracts the S3 trigger configuration for each Lambda function
s3_triggers = {
for k, v in local.lambda_config : k => v.s3_trigger
}

# Derive Lambda environment variables
# This creates a map of environment variables for each Lambda function
lambda_environments = {
for k, v in local.lambda_config : k => v.environment
}

# Derive IAM role names
# This generates unique IAM role names for each Lambda function, including the workspace name
iam_role_names = {
for k, v in local.lambda_config : k => “${k}-lambda-role-${terraform.workspace}”
}
}

Creating an IAM role using iam_role_names local variable# Create IAM roles for Lambda functions
resource “aws_iam_role” “lambda_roles” {
for_each = local.iam_role_names

name = each.value

assume_role_policy = jsonencode({
Version = “2012-10-17”
Statement = [{
Action = “sts:AssumeRole”
Effect = “Allow”
Principal = {
Service = “lambda.amazonaws.com”
}
}]
})
}

Creating Lambda function with lambda_config# Create Lambda functions
resource “aws_lambda_function” “functions” {
for_each = local.lambda_config

filename = “lambda_function_payload.zip” # Ensure this zip file exists
function_name = local.lambda_names[each.key]
role = aws_iam_role.lambda_roles[each.key].arn
handler = each.value.handler
runtime = each.value.runtime
memory_size = each.value.memory_size
timeout = each.value.timeout

environment {
variables = local.lambda_environments[each.key]
}
}

Creating S3 notification and Lambda permissions# Create S3 bucket notification configurations
resource “aws_s3_bucket_notification” “bucket_notifications” {
for_each = local.s3_triggers

bucket = each.value.bucket

lambda_function {
lambda_function_arn = aws_lambda_function.functions[each.key].arn
events = each.value.events
filter_prefix = each.value.filter_prefix
filter_suffix = each.value.filter_suffix
}
}

# Grant S3 permission to invoke Lambda
resource “aws_lambda_permission” “allow_bucket” {
for_each = local.s3_triggers

statement_id = “AllowExecutionFromS3Bucket”
action = “lambda:InvokeFunction”
function_name = aws_lambda_function.functions[each.key].arn
principal = “s3.amazonaws.com”
source_arn = “arn:aws:s3:::${each.value.bucket}”
}

If you want to create more Lambda functions, we can simply create them by adding them to the lambda_config without any fuss. It will allow anyone without Terraform knowledge to add the config create/delete/manipulate Lambda functions, thanks to your map transformation that enabled you to write this flexible Terraform code that scales

You can utilize Terraform functions for more flexibility and error-free code.

Example2: Building Lambda Layers with Terraform and sharing across multiple accounts

I will use the code from my public GitHub Repo that uses the Terraform module to create Lambda layers.

This code is used to automate AWS Lambda Layer creation for Python with Terraform and GitHub Action

I stored the configuration for Lambda layers in local variables.

Used map transformations to create intermittent locals that we will be using to create desired resources.

Creating Layers with Terraform module for Lambda

Adding permissions to allow other AWS accounts to use this Layer.

These two examples explain how useful these map transformations were.

Here are some more use cases of map transformation in

• Copying Maps: If you want to duplicate or clone a map, maintain the same keys and values. This can help in creating independent instances of a configuration.

locals {
  original_map = {
    key1 = "value1"
    key2 = "value2"
    key3 = "value3"
  }

  cloned_map = { for k, v in local.original_map : k => v }
}

• Map Filtering: You can filter out certain elements from a map based on certain conditions, creating a new map with only the elements that meet the criteria.

locals {
  original_map = {
    apple = 5
    banana = 3
    orange = 7
    pear = 2
  }

  filtered_map = { for k, v in local.original_map : k => v if v > 3 }
}

• Map Subsetting: When you want to create a smaller map that contains a subset of the original keys and values, potentially to pass to a module or resource.

locals {
  full_map = {
    a = 1
    b = 2
    c = 3
    d = 4
  }

  subset_map_keys = ["a", "c"]

  subset_map = { for k, v in local.full_map : k => v if k in local.subset_map_keys }
}

• Data Transformation: Converting data formats or structures into a different format, often used in data processing or preparation tasks.

locals {
  raw_data = {
    "name": "John Doe",
    "age": 30,
    "location": "City"
  }

  transformed_data = {
    for key, value in local.raw_data : 
      key == "name" ? "full_name" : 
      key == "age" ? "years_old" : 
      key == "location" ? "city" : key => value
  }
}

• Generating Inputs: Generating input variables or configuration settings for resources or modules based on an existing map of data.

locals {
  resource_settings = {
    "web" = { port = 80, protocol = "http" }
    "db"  = { port = 3306, protocol = "mysql" }
  }

  resource_instances = flatten([
    for name, settings in local.resource_settings : [
      for i in range(2) : {
        name      = "${name}-${i}"
        port      = settings.port
        protocol  = settings.protocol
      }
    ]
  ])
}

• Creating Labels or Tags: When you need to create labels or tags for resources based on predefined keys and values.

locals {
  resource_names = ["instance1", "instance2", "instance3"]
  
  resource_labels = {
    for name in local.resource_names : name => "tag-${name}"
  }
}

• Configuration Composition: When you want to create complex configurations by combining multiple simpler configurations using maps.

locals {
  base_config = {
    region      = "us-west-1"
    environment = "production"
  }

  module_overrides = {
    instance1 = {
      region      = "us-east-1"
      instance_type = "t2.micro"
    }
    instance2 = {
      environment = "development"
      instance_type = "t2.nano"
    }
  }

  final_config = {
    for name, config in local.module_overrides : name => merge(local.base_config, config)
  }
}

These examples should give you a solid foundation for understanding how to perform various map transformations using for loops in Terraform.

Remember that the key strength of this loop construct is its ability to efficiently create new maps that maintain the original structure.

---

Terraform for loop for list comprehension

The same terraform for loop construct can be used for List comprehension as well. Let’s take a few examples of that

• If you want to create a list of keys, or values from a map.

locals {
  aws_bucket_names = {
    "name1" = "bucket1",
    "name2" = "bucket2",
    "name3" = "bucket3"
  }

  bucket_name_list = [for name, _ in local.aws_bucket_names : name]
}

output "bucket_names" {
  value = local.bucket_name_list
}

• Case conversion with terraform for loop

[for s in var.list : upper(s)]

• A for expression can also include an optional if clause to filter elements from the source collection, producing a value with fewer elements than the source value:

[for s in var.list : upper(s) if s != ""]

There can be an infinite number of use cases that you can leverage to write flexible IAC code that is more readable, and scalable.

If you enjoyed the writings leave your claps 👏 to recommend this article so that others can see it

Also, read

If/Else statement in Terraform is much more powerful than you think
How to use the If/Else conditional statement in Terraformmedium.com

Thanks for reading. See you on the next one!