We are still cooking the magic in the way!
Advanced Terraform & IaC Patterns
Dynamic Blocks & Complex Types
Dynamic Blocks & Complex Types
Terraform's type system and meta-arguments scale elegantly from a two-resource demo to a platform engineering codebase managing thousands of resources. Two constructs are central to that scale: dynamic blocks, which let you generate repeated nested configuration from a collection instead of copy-pasting it, and complex types (object, map, list of objects, etc.), which let callers express rich, structured input in a single variable instead of a dozen flat strings. Pair these with Terraform's try and can functions for safe attribute access, and you can write modules that are genuinely flexible without being fragile.
Why Dynamic Blocks Exist
Many AWS resources contain nested blocks that repeat: ingress rules in a security group, lifecycle_rule blocks in an S3 bucket, header blocks in a CloudFront distribution. Without dynamic blocks you must write each one by hand, which means the count is hardcoded in the template. The moment a caller needs one more or fewer rule, they have to fork the module. Dynamic blocks solve this by iterating over a collection and emitting one nested block per element — the same principle as a for-loop, but expressed declaratively inside a resource block.
# Without dynamic blocks — brittle, copy-pasted:
resource "aws_security_group" "web" {
name = "web-sg"
vpc_id = var.vpc_id
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
# Adding HTTPS for IPv6 means editing the module itself — bad.
}
# -------------------------------------------------------
# With dynamic blocks — driven by a variable:
variable "ingress_rules" {
description = "List of ingress rule objects for the web security group."
type = list(object({
from_port = number
to_port = number
protocol = string
cidr_blocks = list(string)
description = optional(string, "")
}))
default = [
{ from_port = 80, to_port = 80, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] },
{ from_port = 443, to_port = 443, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] },
]
}
resource "aws_security_group" "web" {
name = "web-sg"
vpc_id = var.vpc_id
dynamic "ingress" {
for_each = var.ingress_rules
content {
from_port = ingress.value.from_port
to_port = ingress.value.to_port
protocol = ingress.value.protocol
cidr_blocks = ingress.value.cidr_blocks
description = ingress.value.description
}
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
Dynamic block anatomy: The label on
dynamic must match the name of the nested block you want to generate (ingress, lifecycle_rule, etc.). Inside content, use <label>.value.<attribute> to access the current element. Use <label>.key for the map key (or list index). The iterator argument renames the loop variable when the default name conflicts with an existing attribute.Complex Type Constraints
Terraform's primitive types (string, number, bool) are not expressive enough for real module interfaces. Production modules use complex types to enforce structure:
object({...})— a fixed set of named attributes, each with its own type. Use for structured config where every attribute is meaningful and named.map(T)— a lookup table of string keys to values of type T. Use for tag maps, per-service configs keyed by service name, or environment-specific settings.list(object({...}))— an ordered collection of structured items. The canonical input type for dynamic blocks.optional(T, default)— (available from Terraform 1.3) marks an object attribute as optional with a default, making the caller's config much less verbose.
# A realistic module variable using complex types:
variable "services" {
description = "Map of microservices to deploy. Key = service name."
type = map(object({
image = string
cpu = number
memory = number
port = number
desired_count = number
health_path = optional(string, "/health")
env_vars = optional(map(string), {})
secrets = optional(list(string), [])
}))
}
# Caller's terraform.tfvars:
# services = {
# api = {
# image = "012345678901.dkr.ecr.us-east-1.amazonaws.com/api:v2.1.0"
# cpu = 512
# memory = 1024
# port = 8080
# desired_count = 3
# health_path = "/api/health"
# env_vars = { LOG_LEVEL = "info", REGION = "us-east-1" }
# secrets = ["arn:aws:secretsmanager:us-east-1:...:db-password"]
# }
# worker = {
# image = "012345678901.dkr.ecr.us-east-1.amazonaws.com/worker:v1.4.0"
# cpu = 256
# memory = 512
# port = 9090
# desired_count = 2
# }
# }
# Consuming the map with for_each:
resource "aws_ecs_service" "services" {
for_each = var.services
name = each.key
cluster = aws_ecs_cluster.main.id
task_definition = aws_ecs_task_definition.services[each.key].arn
desired_count = each.value.desired_count
}
resource "aws_ecs_task_definition" "services" {
for_each = var.services
family = each.key
cpu = each.value.cpu
memory = each.value.memory
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
container_definitions = jsonencode([{
name = each.key
image = each.value.image
portMappings = [{ containerPort = each.value.port }]
environment = [for k, v in each.value.env_vars : { name = k, value = v }]
secrets = [for arn in each.value.secrets : { name = basename(arn), valueFrom = arn }]
}])
}
Pro practice — use
map(object) over list(object) for uniquely-keyed resources: When driving for_each on a resource, a map gives each instance a stable, meaningful key (the service name, the rule name). A list gives positional indices — reorder the list and Terraform wants to destroy and recreate everything. At Stripe and GitHub, internal platform modules universally prefer map(object) as the primary collection type for resource-driving variables precisely to avoid destroy-on-reorder plan surprises.Dynamic Blocks Inside Dynamic Blocks
Nested dynamic blocks are sometimes necessary — for example, S3 lifecycle rules each contain a nested transition block. Nesting works, but keep it to one level deep in practice; deeper nesting makes the template harder to read than the problem it solves.
variable "lifecycle_rules" {
type = list(object({
id = string
enabled = bool
prefix = optional(string, "")
expiration = optional(number, null)
transitions = optional(list(object({
days = number
storage_class = string
})), [])
}))
default = []
}
resource "aws_s3_bucket_lifecycle_configuration" "main" {
bucket = aws_s3_bucket.main.id
dynamic "rule" {
for_each = var.lifecycle_rules
content {
id = rule.value.id
status = rule.value.enabled ? "Enabled" : "Disabled"
filter {
prefix = rule.value.prefix
}
dynamic "transition" {
for_each = rule.value.transitions
content {
days = transition.value.days
storage_class = transition.value.storage_class
}
}
dynamic "expiration" {
for_each = rule.value.expiration != null ? [rule.value.expiration] : []
content {
days = expiration.value
}
}
}
}
}
# Caller example — zero boilerplate for three tiered rules:
# lifecycle_rules = [
# {
# id = "move-to-ia"
# enabled = true
# prefix = "logs/"
# transitions = [
# { days = 30, storage_class = "STANDARD_IA" },
# { days = 90, storage_class = "GLACIER" },
# ]
# expiration = 365
# },
# ]
Production pitfall — dynamic blocks and the empty collection: If
for_each receives an empty list or map, no blocks are emitted — which is correct. But if the nested block is required by the provider (some resources require at least one ingress rule, or AWS will reject the API call), you must validate upstream that the collection is non-empty. A module that produces a valid Terraform plan but causes an AWS API error at apply time is one of the hardest classes of bugs to diagnose in CI. Add a validation block on the variable that asserts length(var.ingress_rules) > 0 when the resource requires it.Safe Attribute Access with try and can
When working with complex types from external data sources, remote state, or optional object attributes, attribute access can fail at plan time if a key is missing or a value is null. Terraform provides two functions for safe access:
try(expr1, expr2, ...)— evaluates expressions left to right and returns the first one that does not produce an error. Think of it as a try/catch for HCL expressions.can(expr)— evaluates an expression and returnstrueif it succeeds without error,falseotherwise. Designed for use insidevalidationblocks.
# Reading from remote state where a key may or may not exist:
data "terraform_remote_state" "network" {
backend = "s3"
config = {
bucket = "company-tfstate"
key = "network/terraform.tfstate"
region = "us-east-1"
}
}
locals {
# Safely fall back to a default VPC ID if the remote state
# doesn't have the expected output (e.g., first-time bootstrap):
vpc_id = try(
data.terraform_remote_state.network.outputs.vpc_id,
var.fallback_vpc_id
)
# Safely read a nested optional key from a map variable:
# var.config may not always have the "monitoring" sub-key.
monitoring_enabled = try(var.config.monitoring.enabled, false)
retention_days = try(var.config.monitoring.retention_days, 7)
}
# -------------------------------------------------------
# can() in validation blocks — the idiomatic pattern:
variable "kms_key_arn" {
type = string
description = "ARN of the KMS key for encryption. Must be a valid AWS ARN."
validation {
condition = can(regex("^arn:aws:kms:", var.kms_key_arn))
error_message = "kms_key_arn must be a valid AWS KMS ARN starting with arn:aws:kms:."
}
}
variable "subnet_cidr" {
type = string
description = "CIDR block for the subnet."
validation {
# can() wraps cidrhost() — if the CIDR is malformed, cidrhost() throws;
# can() catches that and returns false, triggering the error_message.
condition = can(cidrhost(var.subnet_cidr, 0))
error_message = "subnet_cidr must be a valid CIDR notation."
}
}
# -------------------------------------------------------
# try() for safe map lookups — avoid lookup() with a default:
locals {
# Prefer try() over lookup() for complex nested access:
db_port = try(var.services["database"].port, 5432)
# Reading an attribute that might not exist on older provider versions:
arn = try(aws_lb.main.arn, aws_alb.main.arn)
}
Type Constraints: object vs map vs any
Choosing the right type constraint is a module design decision with real consequences for usability and safety:
- Use
object({...})when the set of attributes is fixed and named — you know exactly what keys exist and each has a specific type. This gives callers named attributes with type checking and IDE completion. - Use
map(T)when keys are caller-defined and arbitrary (a tag map, a feature-flag map keyed by flag name, a per-region config keyed by region code). - Use
list(object({...}))when order matters or the resource expects a specific sequence (e.g., WAF rules evaluated in order). - Avoid
type = anyin public module interfaces. It passes the schema contract check but defers all type errors to apply time, which is far more expensive to diagnose than plan time.
# Putting it all together — a production-grade ALB listener rule module:
variable "listener_rules" {
description = "Ordered list of ALB listener rules. Earlier items have lower priority."
type = list(object({
priority = number
conditions = list(object({
field = string # "path-pattern" or "host-header"
values = list(string)
}))
target_group_arn = string
}))
}
resource "aws_lb_listener_rule" "rules" {
count = length(var.listener_rules)
listener_arn = var.listener_arn
priority = var.listener_rules[count.index].priority
dynamic "condition" {
for_each = var.listener_rules[count.index].conditions
content {
dynamic "path_pattern" {
for_each = condition.value.field == "path-pattern" ? [condition.value] : []
content {
values = path_pattern.value.values
}
}
dynamic "host_header" {
for_each = condition.value.field == "host-header" ? [condition.value] : []
content {
values = host_header.value.values
}
}
}
}
action {
type = "forward"
target_group_arn = var.listener_rules[count.index].target_group_arn
}
}
Key insight — the one-element list trick: When a block must appear exactly once or not at all (optional single block), drive it with
for_each = condition ? [1] : []. An empty list emits no block; a one-element list emits exactly one. This is idiomatic Terraform and far cleaner than duplicating resource definitions. You see this pattern in the dynamic "expiration" example above and in virtually every mature module in the Terraform Registry.