How I Structure a Terraform Module

Introduction

Open a Terraform repository you have never seen before. There is a good chance it looks like this: one main.tf with eight hundred lines in it, a variables.tf where required and optional inputs are shuffled together in alphabetical order, a provider block with no version constraint at all, and a README.md that documented the inputs accurately about four months ago.

None of that is anybody’s fault. Terraform doesn’t care how you arrange your files. Every file ending in .tf in a directory gets concatenated before anything is evaluated, so the language gives you no reason to prefer one layout over another. The tutorials all put everything in main.tf because they’re teaching one resource at a time, and then that becomes the shape of the repo forever.

I’m not a developer by trade. I don’t have the muscle memory that lets someone navigate an unfamiliar codebase by feel. What I have instead is a layout I use every single time, so that six months from now, when I come back to a module I half remember, I know where things are before I open anything.

This post is that layout, and the reasoning behind each piece of it. Every convention here is a response to a specific way I have watched a module go bad. It’s all based on the HashiCorp style guide, with opinions layered on where the style guide leaves room.

Everything below is scaffolded in terraform-module-template, which I use for modules and root modules alike.

TL;DR

• data.tf, locals.tf, providers.tf, terraform.tf, backend.tf, variables.tf, and outputs.tf always exist, even when empty.
• Resources live in main.tf until there are roughly twenty of them, then split by service or function.
• variables.tf is two sections, required before optional, alphabetized inside each. outputs.tf is one alphabetized list.
• terraform.tf holds required_version and nothing else. required_providers belongs in providers.tf.
• Pin everything pinnable with ~> Maj.Min.
• Generate the README with terraform-docs, do not write it.
• Run fmt, validate, docs, tflint, and trivy in pre-commit, so you find problems before a plan does.

Problem 1: Everything Ends Up in main.tf

A module starts with three resources, so of course they go in main.tf. Then it grows. Nobody ever decides to put eight hundred lines in one file. It happens because the alternative requires a decision, and there’s never a good moment to make it.

The Fix: Files by Role First, Size Second

Some file names are reserved for a role, and those files always exist:

backend.tf      state backend configuration
data.tf         data sources
locals.tf       local values
main.tf         resources
outputs.tf      module outputs
providers.tf    required_providers, plus provider configuration
terraform.tf    required_version
variables.tf    input variables

Resources stay in main.tf until there are around twenty of them. Past that, main.tf splits by service or function: network.tf, iam.tf, database.tf. The number isn’t sacred. The point is that the split is triggered by size, and only resources ever get split. Data sources, locals, and provider configuration stay in their own files no matter how few or how many there are.

That asymmetry is deliberate. Resource count varies enormously between modules, so where resources live has to scale. The other categories are small and bounded, and their value comes from being findable at a fixed address rather than from being organized well.

Why Empty Files Stay

In a fresh module, data.tf and locals.tf are empty except for a header comment:

###############################################################################
# data.tf
#
# Contains any data sources
###############################################################################

This is the convention people push back on most, and I’ll keep defending it. An absent file carries no information. You can’t tell whether this module has no data sources or whether someone put them in main.tf without looking. A present, empty file answers both questions at once: there are no data sources yet, and here is where the next one goes.

Most structural decay in a repo doesn’t come from someone disagreeing with the layout. It comes from someone adding a data source at 5pm, not seeing an obvious home for it, and dropping it at the bottom of main.tf. An empty data.tf removes the decision, which is the only reliable way to make a convention survive contact with a deadline.

Problem 2: You Can’t Tell Required From Optional

Alphabetical ordering in variables.tf sounds obviously correct, and it’s the default advice. It also means that the first thing you want to know about a module - what do I have to give it? - is the one thing the file won’t tell you. You have to read every block and check each one for a default.

The Fix: Two Sections, Alphabetized Inside Each

variables.tf is split into required variables, meaning no default, followed by optional variables, meaning there is one. Each section is alphabetized independently:

###############################################################################
# Required Variables (no default values)
###############################################################################
variable "image_id" {
  type        = string
  description = "The id of the machine image (AMI) to use for the server."

  validation {
    condition     = length(var.image_id) > 4 && substr(var.image_id, 0, 4) == "ami-"
    error_message = "The image_id value must be a valid AMI id, starting with \"ami-\"."
  }
}

###############################################################################
# Optional Variables (has a default value)
###############################################################################
variable "instance_type" {
  type        = string
  description = "Instance Type"
  default     = "t3.micro"
}

Now the top of the file is the module’s contract. Everything above the second banner is what a caller must supply. Everything below it is a knob they may choose to turn. Sorting alphabetically inside each section keeps lookup fast without giving up that distinction.

outputs.tf gets plain alphabetical ordering, because outputs have no equivalent of the required/optional split. There is only one axis to sort on, so sort on it.

Make the Generated Docs Agree

A convention that only lives in the source file will drift the moment something else renders the same information in a different order. terraform-docs produces the inputs table in the README, and by default it sorts by name, which would immediately contradict variables.tf.

So .terraform-docs.yml is configured to sort the same way:

sort:
  enabled: true
  by: required

settings:
  anchor: false

Required inputs come first in the generated table, exactly as they do in the file that generated it. It’s a small thing, but it is the difference between a convention that is asserted once in a style guide and one that is enforced in two places and therefore holds.

Problem 3: One File for Every Version Constraint

The common layout is a single versions.tf holding both required_version and required_providers, or a terraform block sitting at the top of main.tf with everything in it. Both work. Both bury two unrelated decisions in the same place.

The Fix: Three Files, Three Concerns

The version of Terraform itself, alone in its own file:

# terraform.tf
terraform {
  required_version = "~> 1.0"
}

The providers, their versions, and their configuration, together in another:

# providers.tf
terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 6.28"
    }
  }
}

provider "google" {
  # Configuration options
}

And where state lives, in a third:

# backend.tf
terraform {
  cloud {
    organization = "example_corp"
    hostname     = "app.terraform.io"

    workspaces {
      tags = ["app"]
    }
  }
}

Yes, that’s three separate terraform blocks across three files. Terraform merges them, so this costs nothing at evaluation time and buys a clean separation:

• terraform.tf is about the tool. It changes when I decide to adopt a new Terraform version, which is a deliberate, infrequent, repo-wide decision.
• providers.tf is about dependencies. It changes when a provider ships something I need, which happens on somebody else’s schedule and much more often. It also holds provider configuration, which is what I’m editing most of the time.
• backend.tf is about where state lives. It’s the one file most likely to differ between environments or to be supplied at init time, and the one most likely to be the reason a terraform init behaves differently on someone else’s machine.

Keeping them apart means a diff tells you which kind of change you are looking at before you read a single line of it.

Problem 4: Pins That Are Too Loose or Too Tight

There are two failure modes here and they look nothing alike.

Too loose, usually meaning no constraint at all, gives you a plan that worked yesterday and doesn’t work today, because a provider released a major version overnight and terraform init on a clean checkout happily took it.

Too tight, meaning an exact pin like version = "6.28.0", gives you a module that never picks up a bug fix unless someone edits it, and a fleet of repos pinned to a scatter of slightly different patch releases.

The Fix: Pessimistic Constraints on Everything Pinnable

Everything that can be pinned gets pinned with the pessimistic operator at the major-minor level: Terraform core, every provider, every module call.

version = "~> 6.28"    # >= 6.28.0, < 7.0.0

Minor and patch releases flow in. A major version, which is where the breaking changes live by convention, does not. That’s the tradeoff I want almost everywhere: I get fixes without being asked, and I get to schedule the upgrades that cost me something.

One edge case. For a provider still on 0.x, ~> 0.104 expands to >= 0.104.0, < 1.0.0, and pre-1.0 providers routinely make breaking changes on a minor bump. If a dependency matters and is still pre-1.0, tighten it to ~> 0.104.0 and accept the manual bumps.

Commit the Lock File

~> 6.28 is a statement about what you will tolerate. .terraform.lock.hcl is a record of what you actually got, and it belongs in version control.

The lock file holds on to a provider entry as long as state still references that provider, even after you’ve deleted every last resource for it from your configuration. The instinct is to open the lock file and delete the stanza. Don’t. Apply first so the resources leave state, and the next terraform init will prune the entry on its own. Hand-editing a lock file to fix a problem that an apply would have fixed tends to produce a second, stranger problem.

Problem 5: The README Drifts

Every Terraform README starts with an accurate inputs table. Then someone adds a variable in a hurry. The code is right and the documentation is wrong, and now the README is worse than no README, because people trust it.

The Fix: Generate It

terraform-docs reads the actual variable and output blocks and writes the tables into the README between marker comments. It runs as a pre-commit hook, so it is not something anyone has to remember:

- id: terraform_docs
  args:
    - --args=--lockfile=false

The one thing that catches everybody the first time: the hook rewrites README.md during the commit, which means the commit fails, because the file it just changed was not part of what you staged. This is correct behavior and it looks like a broken hook. Re-stage and commit again and it passes. It only happens when the generated content actually changed, so after the first time it mostly disappears.

Problem 6: You Find the Error at Apply Time

The expensive version of every mistake in this post is the one you discover after terraform apply has already created four of seven resources.

The Fix: A Pre-Commit Chain, in Order

- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: v1.108.1
  hooks:
    - id: terraform_fmt
    - id: terraform_validate
      args:
        - --hook-config=--retry-once-with-cleanup=true
    - id: terraform_docs
      args:
        - --args=--lockfile=false
    - id: terraform_tflint
      args:
        - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl
    - id: terraform_trivy

The order is cheapest and most mechanical first, so that later hooks are reading code that is already well formed:

1. terraform_fmt normalizes whitespace and alignment. Running it first means no other hook, and no future diff, is ever about formatting.
2. terraform_validate catches syntax and internal consistency errors. --retry-once-with-cleanup=true handles the common case where validation fails only because of a stale .terraform directory: it clears it and tries once more instead of reporting a problem you don’t have.
3. terraform_docs regenerates the README, now that the code it is documenting is known to be valid.
4. terraform_tflint catches the things validate structurally cannot - deprecated syntax, unused declarations, provider-specific mistakes such as an instance type that doesn’t exist.
5. terraform_trivy scans for security misconfigurations. It goes last because it is the slowest and the most likely to need a human judgment call about whether a finding applies.

Alongside those, the standard hygiene hooks run on everything, and one of them earns a special mention:

- id: no-commit-to-branch

That blocks commits directly to main. It’s saved me from myself more than any of the Terraform-specific hooks.

The Directories You Don’t Need Yet

The template ships five directories that are empty apart from a README.md: examples/, modules/, tests/, files/, and templates/.

The placeholder README exists because git won’t track an empty directory, so without a file in it the directory would not survive a clone. But the reason to scaffold them at all is the same reason the empty data.tf earns its place. When it’s time to write the first test, there is no decision to make about where tests go. When someone needs a usage example, examples/ is already there with a note explaining what belongs in it.

Scaffolding an empty directory costs nothing today. Adding one to a repo that has already grown around its absence means moving files and updating every path that pointed at them.

Conclusion

None of this is clever, and that’s the point. It’s a set of decisions made once so that they do not have to be made again while tired:

• Reserved file names, always present - data.tf, locals.tf, providers.tf, terraform.tf, backend.tf, variables.tf, outputs.tf. Empty is fine, and an empty file still tells you something.
• Only resources scale - main.tf splits by service past roughly twenty resources. Nothing else splits, ever.
• Required before optional in variables.tf, alphabetized within each section, with terraform-docs configured to sort the same way so the generated table cannot contradict the source.
• Three files for three kinds of version decision - the tool, its dependencies, and where state lives.
• ~> Maj.Min on everything pinnable, with the pre-1.0 caveat, and the lock file committed as the record of what you actually resolved.
• Generate the README, never hand-maintain it.
• fmt, validate, docs, tflint, trivy in that order in pre-commit, because the cheapest place to find any of these problems is before the commit, and the most expensive is halfway through an apply.

The whole thing is in terraform-module-template if you would rather clone it than assemble it.

If you disagree with a specific convention here, that’s fine and probably healthy. What matters far more than which layout you pick is that you pick one and it’s the same in every repo. Decide once, and you stop spending attention on the question.

Up Next: Terraform in Practice

This kicks off a series on the practices that sit around the Terraform code rather than in it. Next up: getting static credentials out of your infrastructure entirely - a Vault SSH certificate authority instead of distributing key pairs, OIDC instead of long-lived AWS access keys, and dynamic database credentials instead of a shared application user. Three substitutions, and the gotcha that bites on each one.