Peering, Part 3: Overrides, Exclusions, and Importing a Legacy Estate

Part 1 built a generator. Part 2 built the schema that lets the generator stay ignorant of Azure.

Both assumed a uniform estate: every spoke in a region peers to every hub in that region, with default names and default flags. Under that assumption the generator is a much smaller thing and this series is two posts long.

The estate is not uniform. It is old.

The Layout

Everything in this post lives in one directory per subscription, alongside the generated output:

estate/
├── peering_config.json               ← the uniform model: hubs per region
├── bin/generate_peerings.py
└── subscriptions/app-a/prod/
    ├── .env                          ← GENPEER, GENPEER_VERSION
    ├── .peering_overrides.yml        ← the non-uniform reality: this post
    ├── .module_overrides.yml         ← its sibling, from the versioning series
    ├── network.tf                    ← hand-written VNet declarations
    ├── peerings.tf                   ┐ generated on every init,
    ├── peer-providers.tf             │ gitignored,
    └── override.tf                   ┘ never edited by hand

peering_config.json describes the estate as it was designed. .peering_overrides.yml describes it as it was actually built. The gap between those two files is what this post is about.

What Non-Uniform Actually Looks Like

Peerings already deployed in quantity, and:

• The names don’t match a convention. They match several conventions, layered by era — some encode the direction one way round, some the other — and all of them are correct, in the sense that they exist and traffic flows through them.
• The default name isn’t any of them. The module’s default is built from the two VNet names, "${var.vnet_hub.vnet_name}-${var.vnet_spoke.vnet_name}", which matches roughly none of the deployed estate.
• Peering names are immutable. Changing one is a destroy-and-recreate: a connectivity outage between two production networks.
• Some links have deliberate property differences. Some need allow_forwarded_traffic = false, decided long ago, still correct.
• Some spokes must not peer to some hubs. There is usually at least one hub that a given unit has no business reaching.

So the generator has a hard constraint: for every already-deployed peering, it must emit Terraform that matches the deployed resource exactly. Not “equivalent”. Not “better”. Identical, down to the name, so that terraform import produces a no-op plan.

Get that wrong and the plan is a destroy-and-recreate on production connectivity — which, credit where it’s due, Terraform will show you clearly, right before you approve it at 5pm on a Thursday.

Overrides in YAML

The answer was a file next to the subscription’s .tf files, deliberately a sibling of .module_overrides.yml from the versioning series — same pattern, same location, same lifecycle, so there’s one place to look for “what’s weird about this subscription.”

vnets:
  spoke-vnet:
    region-a-hub-vnet-edge:
      hub:
        name: hub-edge-to-spoke-peering
        allow_forwarded_traffic: false
      spoke:
        name: spoke-to-hub-edge-peering
    region-a-shared-vnet-core: {}
  another-spoke-vnet:
    region-a-hub-vnet-edge: {}
    region-a-shared-vnet-core:
      exclude: true

Three levels: which local VNet, which hub it’s peering to, what’s different about that link. Which is exactly the identity of a peering — you cannot address a peering with fewer than two VNet names, and the schema doesn’t pretend otherwise.

Note the {} entries. An empty entry is a no-op. It’s there because a reviewer reading this file should be able to see every hub the spoke connects to and know that the omission of an override for that one is deliberate rather than forgotten. Costs nothing, answers a question that would otherwise require reading the generator.

And note the asymmetry in the first entry: hub and spoke have different names, and the hub also carries a property override. Each direction of an Azure peering is an independent resource with its own name and its own flags. The schema mirrors that, because pretending two things are one thing is how you end up with a _reverse suffix and regret.

The rendering problem

Here’s where the symmetric schema from Part 2 earns back its cost, and where a subtle asymmetry appears anyway.

The hub side is a literal the generator constructs, so an override is just more keys in the map:

    hub_override = ''
    if overrides and overrides.hub:
        hub_content = _format_override_block(overrides.hub)
        hub_override = f"""
    override = {{
      {hub_content}
    }}"""

  vnet_hub = {
    subscription_id     = "00000000-0000-0000-0000-000000000000"
    resource_group_name = "region-a-hub-rg"
    vnet_name           = "region-a-hub-vnet-edge"
    fail_if_not_found   = true
    override = {
      name                    = "hub-edge-to-spoke-peering"
      allow_forwarded_traffic = false
    }
  }

The spoke side is not a literal. It’s module.spoke-vnet.peering — an object produced by another module, at apply time. You cannot add a key to that from the outside. You can only merge:

    if overrides and overrides.spoke:
        spoke_content = _format_override_block(overrides.spoke)
        spoke_line = f"""merge(module.{module_ref}.peering, {{
    override = {{
      {spoke_content}
    }}
  }})"""
    else:
        spoke_line = f'module.{module_ref}.peering'

  vnet_spoke = merge(module.spoke-vnet.peering, {
    override = {
      name = "spoke-to-hub-edge-peering"
    }
  })

merge() is evaluated at apply time, so it works fine with values that are unknown at plan time — the whole point of module_output from Part 2. The generator emits a transformation rather than a value, and Terraform performs it once the values exist.

The else branch matters as much as the if. With no spoke override, the output is the bare reference — no merge() wrapper, no cosmetic noise. Generated code that gets more complicated when it doesn’t need to be is generated code people stop reading.

Booleans, quoting, and the small indignities of codegen

def _format_override_block(overrides: dict[str, str | bool]) -> str:
    """Format override dict as Terraform override block content.

    Handles str values (quoted) and bool values (lowercase unquoted).
    """
    if not overrides:
        return ''
    lines = []
    for k, v in overrides.items():
        if isinstance(v, bool):
            lines.append(f'{k} = {str(v).lower()}')
        else:
            lines.append(f'{k} = "{v}"')
    return '\n      '.join(lines)

YAML gives you a Python True. HCL wants true. str(True) is "True", and allow_forwarded_traffic = True is not valid HCL — it’s an unquoted identifier reference to a variable that doesn’t exist, and the error message will talk about undeclared references rather than about your YAML.

Every code generator that crosses a language boundary contains a function like this, and every one of them was written after the bug. The module configurator’s render_hcl_value is the same function, grown up, with lists and nested objects and a _raw escape hatch. This one stayed small because the override schema is deliberately closed: four keys, strings and booleans. When a schema can’t grow arbitrarily, its serialiser doesn’t have to either.

Undoing A Design Decision

Overrides went in and worked. Exclusions were the obvious next thing, and I put them in the wrong place first.

They originally lived in the Terraform module, as a variable:

variable "exclude_peerings" {
  description = "A list of VNET names to not peer with."
  type        = list(string)
  default     = []
}

Which is defensible. It’s next to generate_peerings, it’s in the module you’re already configuring, it reads naturally.

It was wrong for four reasons, and I’d like to flag the first one because it’s the one you can spot in your own repo today. Look at the sibling variable’s description:

variable "generate_peerings" {
  description = "A variable when set to true, will trigger the peer generator wrapper for this VNET. Otherwise unused in this module."
  type        = bool
  default     = false
}

“Otherwise unused in this module.”

That’s a variable in a published, versioned Terraform module that the module ignores. It exists to be read by a Python script that scans the file as text. It is not configuration for the VNet — it’s an annotation on the VNet, for a different tool, wearing configuration’s clothes.

One such variable is a pragmatic hack. Two is a pattern, and a pattern that says the module’s public interface is drifting into being a config file for something else. generate_peerings earns its place — it’s a genuine per-VNet property, it’s boolean, it’s the on-switch. exclude_peerings didn’t:

It split the config. Names in YAML, exclusions in HCL. Answering “what’s special about this subscription’s connectivity?” meant reading two files in two languages, and knowing that this half lives here and that half lives there.

Terraform validated nothing. list(string). Any string. A typo’d hub name is a valid list(string) — the exclusion silently doesn’t apply, the peering gets generated, and you find out when a plan proposes a peering you thought you’d excluded. Or, worse, you don’t find out.

Changing an exclusion was a module input change. Which is a real Terraform diff, in the primary .tf file, mixed in with genuine infrastructure changes.

The unmanaged module had to carry it too. Both VNet modules needed the variable, so it lived in a shared file symlinked into both — which, as the versioning series covered, means editing it is a change to every module that links it.

So I deleted the variable and moved the capability into the YAML, where the rest of the per-link config already lived:

    region-a-shared-vnet-core:
      exclude: true

                override = module_overrides.get(peering.vnet_name)
                if override is not None and override.exclude:
                    continue

One dict lookup and a continue. The exclusion is now keyed by the same (module, hub) pair as every other override, so it can’t be typo’d into silence — because of what shipped alongside it.

The linter

Moving config out of a typed language means the type checker is gone. So it gets rebuilt, at the only place that can do it: the generator, which is the one component that knows both the YAML and the resolved hub list.

def lint_overrides(all_overrides, modules, hubs) -> list[str]:
    """Check every override target resolves to a known hub for the spoke's peer region.

    Returns a list of error messages (empty when clean). Caller is expected to
    exit non-zero when any errors are returned.
    """
    errors: list[str] = []
    for module_name, targets in all_overrides.items():
        module_data = modules.get(module_name)
        if module_data is None:
            errors.append(
                f'Override references unknown module {module_name!r} '
                '(no network or unmanaged-network module by that name was found).'
            )
            continue
        region = module_data.get('region')
        if region is None or region not in hubs:
            errors.append(
                f'Override for module {module_name!r} cannot be linted: '
                f'region {region!r} not present in peering_config.json hubs.'
            )
            continue
        resolved_hub_names: set[str] = set()
        for vnet_list in hubs[region].values():
            for vnet in vnet_list:
                effective_region = vnet.hub_region or region
                resolved_hub_names.add(
                    vnet.name.replace(REGION_PLACEHOLDER, effective_region)
                )
        for target_vnet in targets:
            if target_vnet not in resolved_hub_names:
                errors.append(
                    f'Override for {module_name!r} references unknown hub '
                    f'{target_vnet!r} in region {region!r}. '
                    f'Known hubs: {sorted(resolved_hub_names)}'
                )
    return errors

Three failure modes, three distinct messages, all naming the thing that’s wrong and — in the third case — listing every valid alternative. Known hubs: [...] turns a five-minute grep into reading the error.

Note that it lints against resolved names, applying the same REGION substitution and hub_region override the generator itself uses. A linter that validates against a different model than the one that generates is a linter that will eventually disagree with reality and be ignored.

It exits 2, distinct from the 1 used for config load failures, so a pipeline can tell “your YAML is wrong” from “the config file is broken.”

And one warning rather than an error:

            if exclude and (hub or spoke or options):
                logger.warning(
                    'Override for %s -> %s sets exclude:true but also defines '
                    'hub/spoke/options — exclude wins, sibling overrides are dead.',
                    module_name, target_vnet,
                )

Not an error, because it’s unambiguous — exclude wins, no data loss, no wrong infrastructure. But it’s nearly always a mistake in progress: someone excluded a peering to deal with an incident and left the carefully-worked-out names underneath, and in three months someone will remove the exclude and get names they didn’t review. The warning says which rule won, in the same words as the docs. Precedence rules that are only documented aren’t rules; they’re trivia.

Importing The Estate

The reason all of the above exists is a single piece of work: bringing already-deployed peerings under management without touching a single one of them.

The workflow, and it’s more mechanical than it sounds:

1. Enumerate what exists. az network vnet peering list across the hub subscriptions, dumped to a file. Worth marking a script like that as ad-hoc in whatever way your repo does, because the alternative is pretending every throwaway is production.

2. Write the YAML from reality. Not from the convention. Every peering’s actual deployed name becomes a hub: or spoke: name override. This is the tedious step and it is the whole job.

3. Generate and compare. --dry-run prints without writing:

    parser.add_argument('--dry-run', action='store_true',
                        help='Print generated output to stdout instead of writing files')

4. Import. terraform import each peering into its generated address.

5. Plan until it’s empty. An empty plan means the generated Terraform describes what is actually deployed. Not equivalent — identical.

Step 5 is the acceptance test, and it’s the only one that matters. Every discrepancy is a bug in step 2 or a missing capability in the generator — and allow_forwarded_traffic: false in the override schema is exactly that: a property override that exists because deployed peerings had it and the plan wouldn’t go empty until the generator could express it.

The empty plan is also the moment the estate stops being legacy. Same generator, same YAML, same review process for a peering deployed long before any of this existed as for one created tomorrow.

The rebuild switch

One more operational mode, from the same era:

    parser.add_argument('--no-fail-on-missing', action='store_true', default=False,
                        help='Set fail_if_not_found=false for all peerings (useful during rebuilds). '
                             'Also settable via GENPEER_FAIL_ON_HUB_NOT_EXIST=False env var.')

    force_no_fail = args.no_fail_on_missing or os.environ.get('GENPEER_FAIL_ON_HUB_NOT_EXIST', '') == 'False'
    if force_no_fail:
        logger.info('force_no_fail enabled: all peerings will have fail_if_not_found=false')

Normally a peering onto a managed estate gets fail_if_not_found = true: the hub should exist, and if it doesn’t, stop. But during a region build the hubs don’t exist yet, and you want the spokes to deploy anyway and pick up their peerings on a later pass.

The flag flips every peering to the discovery path — the safe, slow, N+1 one from Part 2. It’s deliberately expensive, and it announces itself in the log every run, because a mode that suppresses your safety net should never be quiet about it. Leave it on permanently and every peering silently skips instead of failing, which is exactly the failure the safety net exists to prevent.

A CLI flag and a real environment variable — not a .env key, because dotenv_values reads that file without touching os.environ. The env var is the one you reach for when the script runs from a hook and there is no argv to control (the versioning series covers why).

Two Generators, One Hook, Order Matters

Worth closing on, because it’s the seam between the two series and it’s the kind of thing that bites.

terraform {
  before_hook "peering_generation" {
    commands = ["init"]
    execute  = ["${get_repo_root()}/bin/generate_peerings.py"]
  }
  before_hook "module_overrides" {
    commands = ["init"]
    execute  = ["${get_repo_root()}/bin/module_configurator.py"]
  }
}

The peering generator emits peerings.tf containing module blocks whose source is the marker git::https://PEERING_MODULE. The module configurator then scans every .tf file — including the freshly generated peerings.tf — and resolves markers to pinned versions in override.tf.

So the generated peering modules are version-pinned by exactly the same manifest as everything else. No special case, no second version list. The peering generator doesn’t know what version of the peering module exists, and doesn’t need to.

That ordering is load-bearing and nothing enforces it beyond the order of two blocks in an HCL file. Swap them and the configurator runs before peerings.tf exists, resolves no peering markers, and terraform init fails trying to clone git::https://PEERING_MODULE. Which — I’ll be honest — is at least a loud failure, and I’d rather have this dependency loud and undocumented-in-code than clever and silent. So if you build this pattern, put a comment there saying so — which is what the header of the shipped hooks.hcl does.

There is an escape hatch for pinning a peering module version out-of-band:

    module_version = args.module_version or dotenv.get('GENPEER_VERSION', None)

def _build_source_url(module_version: str | None = None) -> str:
    base = 'git::https://PEERING_MODULE'
    return f'{base}?ref={module_version}' if module_version else base

An explicit ?ref= in the marker wins over the manifest — the same escape hatch Part 2 of the versioning series describes, reused rather than reinvented. One subscription can trial a new peering module version without touching the estate-wide manifest.

What To Steal

Design the module for the generator, not the human. The symmetric vnet_hub/vnet_spoke schema and the peering output are why the generator is small. If your generator is doing string surgery to build resource IDs, the module’s interface is wrong.

Publish an output whose shape is another module’s input. That’s a contract between modules, versioned by tags, that the generator only has to reference rather than understand.

A module that creates nothing is a legitimate module. network_unmanaged exists so that things outside your control can be referenced by the same automation as things inside it — and it turned legacy adoption into a one-file swap.

Don’t put codegen annotations in module variables. If a variable’s description contains “otherwise unused in this module,” it belongs in a config file next to the code that reads it.

When config leaves a typed language, rebuild the type checker. Lint it in the tool that consumes it, against the resolved model, and put the valid options in the error message.

Make the acceptance test “the plan is empty.” For any migration onto generated infrastructure, that’s the only criterion that means anything. Not equivalent. Empty.

Generate for the estate you have. A generator that only produces the convention is a generator for a greenfield estate, and you don’t have one of those. The override file isn’t debt — it’s an accurate description of years of decisions, and having it in one reviewable YAML file per unit is enormously better than having it scattered across every hand-written resource.

Several rewrites, one retraction, and a legacy estate that now plans empty.