Pymantic Release, Part 3: Wrapping It All in Terragrunt

Part 1 built a release pipeline that publishes correct per-module tags. Part 2 built a manifest and a generator that resolves those tags into every subscription in the estate.

And then ended on the admission that the whole thing hinges on somebody typing python ../../bin/module_configurator.py in the right directory before they type anything else.

That’s not a versioning system. That’s a versioning system plus a ritual, and rituals decay. Somebody joins the team and doesn’t know about it. Somebody’s shell alias breaks. Somebody runs terraform plan directly out of muscle memory built over nine years, because terraform plan is a real command that exists and nobody’s brain has ever accepted otherwise.

The Layout

The relevant half of the consumer repo, since everything below hangs off it:

estate/
├── module_versions.json
├── bin/
│   └── module_configurator.py
└── subscriptions/
    ├── ...                  ← provider, state and tag config
    ├── hooks.hcl            ← the six lines this post is about
    └── app-a/prod/
        ├── terragrunt.hcl   ← `include` blocks only
        ├── .env             ← everything that varies, per unit
        ├── bootstrap.tf     ┐
        ├── network.tf       ┴ hand-written, with markers instead of versions
        ├── override.tf      ┐
        ├── provider.tf      ├ generated on every init, all gitignored
        ├── versions.tf      │
        └── tags.tf          ┘

Subscriptions nest at different depths — some are <business>/<env>/, some are <business>/<env>/<name>/ — which matters more than it looks like it should.

The Load-Bearing Six Lines

Terragrunt has hooks. Hooks run before or after a given Terraform command. That’s the entire feature and it is worth the whole dependency:

# subscriptions/hooks.hcl
terraform {
  before_hook "module_overrides" {
    commands = ["init"]
    execute  = ["${get_repo_root()}/bin/module_configurator.py"]
  }
}

Two things to notice, and a third the block doesn’t show.

get_repo_root(). Not a relative path. The configurator can be invoked from subscriptions/app-a/preprod/ or subscriptions/platform/shared/ — nested at different depths — and resolves the same way from both. Terragrunt sets the working directory to the unit being run, so the script’s own getcwd()-based scanning does the right thing without knowing where it is. The pairing is the point: absolute path to the script, relative behaviour once it’s running.

commands = ["init"]. Not plan. Not apply. Terragrunt runs init automatically before plan and apply when it needs to, so hooking init covers every path into the configuration exactly once. Hooking plan as well would regenerate the file mid-workflow, and a file that changes between plan and apply is how you get an apply that doesn’t match its plan. Once, at the front.

Hooks compose, and the order is a property of the file. Several before_hook blocks on the same command run top to bottom, in the order written. That is fine while they are independent and a trap the moment they are not — if a later generator has to read what an earlier one wrote, that dependency lives nowhere except the sequence of two blocks in an HCL file, and nothing enforces it. Swap them and you get a failure that talks about a missing module rather than about ordering. When you reach that point, write the comment; nobody will infer it.

What Actually Happens When You Run A Plan

A unit’s terragrunt.hcl is nothing but include blocks — provider, state, tags, and the one that matters here:

... provider, state and tag includes ...

include "hooks" {
  path = find_in_parent_folders("hooks.hcl")
}

Nothing in it is specific to the unit. Everything that varies lives in a .env file next to it. So terragrunt plan:

1. Resolves the includes, walking up to the shared config.
2. Reads .env for whatever varies per unit — here, SKIP_MODULES.
3. Generates its own files — provider, versions, tags — from those includes.
4. Runs init, which fires the before_hook and produces override.tf.
5. Runs the plan, against a configuration that is now fully resolved.

The .gitignore tells the story better than I can:

# Terragrunt Generated
**override.tf
**versions.tf
**provider.tf
**tags.tf

Four generated files. None committed. None reviewed, because reviewing generated output is a category error — you review the generator and the inputs. The inputs are module_versions.json, .env, .module_overrides.yml, and the human-written .tf files with markers in them. Everything else is derived, deterministic, and disposable.

Which is what closes the loop from Part 2. It’s no longer possible to forget to run the configurator, because there is no command you’d run instead. terragrunt plan regenerates it. terragrunt apply regenerates it. CI regenerates it. Delete override.tf out of spite and the next command puts it back.

A Note On What Terragrunt Is Not Doing Here

We are not using Terragrunt’s headline feature. There’s no terraform { source = "..." } block, no dependency graph between units, no dependency blocks feeding outputs between subscriptions.

That’s deliberate. Terragrunt’s source-and-inputs model would replace module_versions.json with a Terragrunt-flavoured version of the same idea, and pull module version resolution into HCL where it’d be considerably harder to test than 339 lines of Python with a pytest suite. The markers-and-override.tf approach also degrades gracefully: run terraform directly with a stale override.tf on disk and you get a working, if possibly outdated, configuration. Depend on Terragrunt to fetch your modules and Terraform alone can’t do anything at all.

So Terragrunt is used for four things, all of which it’s genuinely best-in-class at: include-based config inheritance, remote state key computation, file generation, and hooks. Module versioning stays in Python, where it can be unit tested. Given the number of ways Part 1 established that this logic can be subtly wrong, “where it can be unit tested” is not a small consideration.

The Portability Test

Here’s the bit that makes the whole thing worth writing up, and it wasn’t planned. It was discovered, when we had to do all of this again for AWS.

The same two scripts now run on both clouds: pymantic-release.py in each module repository, module_configurator.py in each repository that consumes them. Same file names, same manifest shape:

{
    "NETWORK_MODULE": {
        "module": "virtual_private_cloud",
        "version": "virtual_private_cloud/v1.14.0"
    },
    "ROUTE_TABLE_MODULE": {
        "module": "route_table",
        "version": "route_table/v1.0.0"
    }
}

Different cloud, different module names, identical machinery. Porting the configurator is a change of one value — the repository it points at:

module_repository = environ.get('MODULE_REPOSITORY', '...')

# module_versions.json
{
  "NETWORK_MODULE": {
    "module":  "virtual_network",
    "version": "virtual_network/v1.14.0"
  },
  "SUBNET_MODULE": { … }
}

# module_configurator.py
MODULE_REPOSITORY=example-org/networks-azure-modules

Because none of it knows anything about Azure. Pymantic knows about git tags and file paths. The configurator knows about HCL module blocks and a JSON lookup table. Neither has ever heard of a virtual network. Nothing imports azure-mgmt-network, nothing calls an ARM API, nothing special-cases a resource type. The Azure-specific knowledge is entirely in the Terraform, where it belongs.

I’d like to claim foresight. Honestly it’s the natural consequence of writing the tool against the actual constraint — Terraform resolves modules by git ref from a subdirectory — which is a property of Terraform, not of any cloud provider.

The Whole Loop

Standing back, here’s the system:

Two human decisions in the entire chain. “This change is a Feature” — expressed in the PR title, at the moment the author has the most context they will ever have about it. And “this estate adopts v1.8.0” — expressed as a one-line diff to a JSON file, reviewed by someone who can see exactly what it changes.

Everything else is derived. The version arithmetic is derived. The relevance is derived. The URLs are derived. The override.tf is derived, regenerated on every init, and gitignored so nobody ever has to look at it.

What I’d Tell You To Steal

If you’re staring down a Terraform module monorepo:

Version the artefact, not the repository. The tag is <module>/v<x.y.z> because the consumer pulls //modules/<module>?ref=<tag>. Make your version scheme match your consumption boundary exactly, and half the hard problems stop existing.

Relevance is about file paths, not commit messages. Every off-the-shelf tool gets you the commit-message half. The half that actually prevents fifty-seven pointless releases is “did this commit change bytes that ship?”, and only you can answer that, because only you know your repo layout. Anchor the check (startswith), don’t float it (in).

PR mode and publish mode must be the same code path. One boolean flag, one calculation. The instant your preview diverges from your action, people stop reading the preview, and then you’ve got a rubber-stamp gate that is worse than no gate because it looks like assurance.

Generate; never rewrite in place. override.tf beats sed -i for one reason that outranks all others: a cancelled pipeline leaves a clean working tree. Terraform’s override-file mechanism exists precisely for this and almost nobody uses it deliberately.

Put the generator behind a hook. A generated file whose generation is a documented step in a README is a generated file that will, on some Tuesday, not be generated. before_hook on init.

Fail loudly on misconfiguration. The unknown-instance linter in the configurator exits 1. Tools that silently ignore your typos are how you end up debugging a plan for forty minutes before discovering the override you wrote was never applied.

Test the path-matching. Not the happy path — nobody gets modules/network/main.tf wrong. Test that examples/ doesn’t count, that network doesn’t match network_gateway, that v1.10.0 sorts above v1.9.0, that a symlink into a provider cache doesn’t crash the walk. Every one of those was a production bug before it was a test, and the tests take 40 milliseconds.

Two years, six repos, two clouds, and three genuinely embarrassing bugs.

Would recommend. Mostly.