What Golden Paths Actually Are (and Why Most of Them Die)

Across my career as a DevOps engineer, I've rebuilt the same thing more than once. New company, new team, and the first months look familiar: stand up CI, wire up deployment to Kubernetes, sort out secrets, decide how config and ingress work, figure out where environments come from. I built it at the last place, and here I am building it again — not because the knowledge was missing. The know-how came with me every time: the standards, the opinions, the scars from doing it wrong before. What didn't come with me was the work. There was no traversability between teams and companies — no template to clone, no shared platform to land on, no artifact of the previous build I could reuse. The opinions transferred; the implementation didn't. So the hard work started from zero, every time, built from the ground up. A golden path is what makes the work transfer, not just the knowledge.

I run GitOps with Argo CD across 10 clusters and roughly 110 applications now, and I've stopped being willing to solve the same problem twice. These days I build what the industry calls golden paths — and this post kicks off a series where I'll show exactly how I build them, baseline by baseline, with the real YAML and the real breakage. Not the conference-talk version.

What a golden path actually is

The term got popularized by Spotify around 2014, and they borrowed it from Frank Herbert's Children of Dune — the one future where humanity survives. Their definition is the one worth stealing: a golden path is the opinionated and supported way to build something.

Both words carry weight, and most failed attempts I've seen drop one of them.

Opinionated means the path makes decisions for you. Which CI system, which deployment mechanism, where the config lives, how secrets get in. Every decision the path makes is a decision an engineer doesn't have to research, debate, and get wrong three times before shipping.

Supported means if you're on the path and something breaks, someone owns the problem. There's a team, a channel, a runbook. This is the part that turns a wiki page into a path.

And here's the clause that makes it all work politically: you can leave the path whenever you want. Nobody's stopping you from running your own deployment stack. But you're off the path, which means you're off the support. Your 2am page, your migration, your problem. This isn't punishment — it's just an honest contract. Autonomy is preserved, but autonomy has a price tag, and most engineers, most of the time, happily choose the paved road.

A golden path is not a standard. Standards are mandates enforced by review boards and they breed resentment and workarounds. A golden path wins by being better, and it has to keep winning, or people route around it.

Who actually uses them

Three audiences, and they want different things.

New hires are the obvious one. A golden path turns "spend your first three weeks asking questions" into "ship something real in your first week." Spotify's data backs this — their golden path tutorials are the most-read documentation in the company because every new engineer goes through them.

Experienced engineers crossing into unfamiliar territory get less airtime but matter just as much. A backend engineer who needs a data pipeline, an app developer who suddenly needs a database with real backup semantics. They're not junior — they just don't want to become a part-time expert in your corner of the stack to accomplish one task.

The platform team. This is the one nobody says out loud, so I'll say it: the platform team benefits the most. Every engineer off the path is a snowflake you'll eventually get paged about, or worse, a migration you'll have to negotiate team by team. My ~110 Argo CD apps are manageable because they share shape — same sync policies, same project structure, same conventions. If half of them were hand-rolled special cases, I'd spend my week on archaeology instead of building.

Being honest about this matters. Engineers smell it instantly when you frame a golden path as purely for their benefit while it's obviously also your maintenance strategy. Own it: the path makes your life easier and my life easier. That's not a hidden agenda, it's the deal.

How they usually start

The mythology is Hack Week. Spotify's first golden path tutorial — backend engineering — came out of a Hack Week project where eight engineers wrote a step-by-step walkthrough of the recommended way to use their services. That's the honest pattern: pain gets loud enough that someone writes down the blessed way to do one thing, and it spreads because it works.

The failure mode is the opposite: a platform team decides to build the full Internal Developer Platform up front. Portal first, templates later, adoption never. I've watched that movie from a distance and I know how it ends — a beautiful Backstage instance with three catalog entries, one of which is a test.

Mine started much smaller. The first thing I built that I'd retroactively call a golden path wasn't Backstage and it wasn't a tutorial. It was one Crossplane XRD and a cdk8s template that replaced a wiki page explaining how to request infrastructure. The wiki page was the pain: it was wrong half the time, and following it still ended in a ticket queue. The XRD turned "read the page, open a ticket, wait" into "apply this claim, get the thing." Once that worked, the pattern repeated — another XRD, another template, and only then a Backstage instance to make the paths discoverable.

That's the order that works: path first, portal second. The portal catalogs paths; it can't create them. If you can't describe the blessed way to deploy a service in a few sentences, no software template will save you.

Why most of them die

Because nobody owns them after the launch. A golden path is a product with users, and products without owners rot. Tool versions drift, a step stops working, the tutorial still references the pipeline you deleted six months ago — and the first engineer who hits a broken step learns the exact wrong lesson: don't trust the path, build it yourself. You're back to from-scratch everything, except now with a graveyard of stale templates on top.

Spotify's fix is worth copying in spirit: the golden path tutorials have special status. If a team owns part of one and has limited time for docs, that time goes to the golden path first. In my world the enforcement is simpler — the paths are code. The XRD is the documentation. If the cdk8s template doesn't compile, CI is red. You can't have a stale template when the template is a build artifact. You absolutely can have a stale tutorial, which is why the tutorial layer of my paths is thin and points at working code.

The series

This is the kickoff. The rest of the series is the "how," one baseline at a time, all from things I run:

• Infrastructure as a golden path — Crossplane v2 XRDs as self-service APIs: what goes in the XRD, what stays out, and where the composition boundary bit me
• Cloud account vending — provisioning development cloud accounts for teams as a self-service path, with guardrails instead of ticket queues
• Cluster as a Service — the golden path for getting a Kubernetes cluster: what the team specifies, what the platform owns, and how it lands in GitOps automatically
• Scaffolding — cdk8s and Backstage templates that generate a service that's already on the GitOps path
• Secrets as a golden path — External Secrets + SOPS under a hard constraint: no cloud secret stores allowed, so the path has to work fully self-hosted
• CI baseline — reusable GitHub Actions workflows so a new repo gets lint, build, and release for free instead of a copied-and-drifted pipeline
• Deployment — the Argo CD app-of-apps structure behind 110 apps on 10 clusters, and the E2E tests on kind that keep the path honest
• Data — the paved road for stateful workloads: CNPG PostgreSQL with WAL archiving that I actually restore-test
• Observability paved road — Grafana dashboards and alerts as code, provisioned with the platform instead of hand-built per team

Each post follows the same deal: what I ran, what broke, the number, the config.

One warning before you start building your own: don't build a golden path for something nobody has asked for. Your backlog is written in repetition — every question you've answered twice, every deployment you've hand-rolled more than once is a path waiting to be paved. Start with the one that hurts most, not the one that demos best.