Turns out I'm Not That Incompetent: Atlas & Validation of the Design

I make a living writing software. Lately I've also been speaking with machines to make them write code, which is a different sport entirely. And these LLM agents altered the rules of this sport for us in a big way.

The company I work for runs on vast stack of code. A pile of packages written by us. The libraries they share. The tests that cover them. Each package moves on its own clock. Each ships on its own schedule. And every one of them depends on the others.

That setup is normal for a platform our size. Then you add LLM agents to it.

An agent is fast. Genuinely fast. It also wakes up knowing nothing about your system, and it will happily invent a pattern that already exists three files over, or lean on an interface somebody is halfway through rewriting. With total confidence. Confidence is free for them.

Every session is a new Project Hail Mary for them. Amnesia, alone, reverse-engineering the world from scratch.

Left to guess, it fills in the blanks. Sometimes the blank is harmless and you ship a duplicate. Sometimes the blank was load-bearing and you find out at 5pm. On a Friday.

We got tired of that. So we built Atlas.

The problem is older than the agents

It's a parallel-work problem. It has been around since 2 developers first touched the same codebase at the same time. Agents just turned the volume all the way up.

The failures are old. Architectural drift, when the code stops following the patterns everyone agreed on because someone improvised. Branch collisions, when parallel work overwrites a dependency. Opaque dependencies, when a developer guesses which interface is safe to build on. Undocumented features, when code ships with no promise behind it.

LLM agents poured gas on all of it. They write code faster than any human team can review, and they wake up knowing nothing about your systems until you feed them context.

4 months on my own machine

I'm not going to pretend I saw the whole shape on day one. I didn't. I ran a local version of this system on my own machine for 4 months, just to survive the problem. Duplicated work, broken dependencies, the works. I got tired of it before I got clever about it. That's usually the order of operations for me.

Then a younger colleague of mine (really good and eager to learn) and I ended up working on features that needed each other. You can't scale agents across multiple packages when work in one repo quietly breaks another; they (LLM agents) like to be creative to the point of leaving Barney with its mouth open in disbelief at the things they dare to invent. We both agreed on that. So I showed him what I had been building for myself, locally. It clicked for him immediately (I want to believe that). We took the thing I had been running solo, polished the data model together to make it useful for a team (no longer a solo venture), and Atlas is what came out.

Then Cloudflare published Codex

About a month into using Atlas internally as a team, Cloudflare published Codex. Same bones: governed standards, agent-consumed context at the point of work, a fixed status for every rule.

We had been walking the same path with a much smaller team, and we got there without the engineering budget of a 3000-person company.

When a software platform and a giant infrastructure company arrive at the same design on their own, the problem only has one shape. Reading their post felt like validation. I've written before about the impostor that never leaves my side. This time he had to sit down and watch in silence. Turns out I'm not that incompetent. Some days. This was one of them.

What Atlas actually is

Atlas is a ledger and a constraint layer. It's a machine-readable model of the architecture, and the tooling checks it against the actual code, locally and in CI.

The core unit is a contract: a written promise about a piece of code. Its name. What it does. Whether other code is allowed to depend on it. Every contract carries a status, and the status is the whole game.

A contract is stable when it's safe to build on. It's unstable when it's about to change, so you ask first. It's editing when somebody is in there right now. That status is the difference between "build away" and "hold on, this is moving."

We type every contract. Some as php-symbol, rest-route, js-export, db-model, etc. A PHP-symbol-only system could never work for us, because the stack lives on hooks and REST routes, and those are strings. So we typed all of them.

2 places we went our own way

First, we made ASD-STE100 the documentation language. That's the aerospace controlled-vocabulary standard: short sentences, active voice, a closed set of approved words. Yes, NASA-grade documentation for a CRUD systems. I hear how that sounds. It works.

Standards need a language, not just a schema. Ambiguous prose burns tokens and makes an agent behave differently on every run. Written in STE100, a document reads identically to a human and to a model. It's cheap to load and hard to misread.

And it sets hard limits. Sentence length is capped, and so is the text and YAML in every document. No document in Atlas ever grows into an encyclopedia nobody will read or maintain.

Second, we resolve typed contracts against the actual code. Most setups extract a prose statement and check the prose. We take a typed identifier, a php-symbol or a rest-route, and resolve it against the codebase. The tooling tells you whether the documented surface still exists. That's a strictly stronger check.

How it runs

The rule is simple. You write the plan before the code. Agents use predefined templates (as schemas) depending of what we are working on, fill in the contracts, set the status. Before you touch a path, you run the collision check. If your path crosses a contract someone else is editing, the check tells you before you write a single line.

When the work ships, the plan's rationale moves into an immutable Architecture Decision Record. The plan itself goes to the archive. The decision stays locked, dated, and signed. We stop relitigating it.

Enforcement comes from atlas lint, a standalone CLI binary. It walks every documented surface and resolves it against the real codebase with real PHP 8.2+ AST traversal. Document a hook that no longer exists, and CI fails. List a dependency that has no package doc, and CI fails. It has a --fix mode that moves each document into the folder its status requires, rebases the links, and repoints everything that pointed at the old path.

Output is SARIF, the standard format GitHub renders as inline annotations.

The binary does a handful of jobs, and each one is a command you can run. lint scans the knowledge base for drift, broken links, orphaned docs, dependency cycles, and style. contract takes a single plan and resolves every surface it promises against the code. check is the collision detector: point it at the files you're about to touch, and it names the active plans editing or destabilizing the same surface. review runs the same checks over one plan, one postmortem, or one package's public surface. explain just tells you, in plain English, what a rule wants and how to fix it. Run any of them locally, in a pre-commit hook, or in CI.

One rule holds the engine together: it never calls a model. Every check is AST parsing, YAML schemas, git timestamps, and exact set intersections. The same input gives the same verdict in a pre-commit hook on your laptop and in CI on GitHub. Determinism is the whole point, so the probabilistic heuristics stay out.

The agent loop

The agents don't wait to be told. Two things make them proactive. Another internal CLI tool assembles the dev stack the exact same way every time, so every agent session starts in an identical environment. One shared AGENTS.md, an instructions file each agent reads on startup, tells every session where to start: Atlas first, before the first keystroke.

Then the agents use atlas lint themselves. They run it to validate their own work, and they write back to Atlas as they go: new quirks, data maps, contracts, relationships, plans, and the rest.

Every one of those writes gets auto-linted. A broken reference fails. A documented surface that doesn't exist in the code fails.

The agents can't ship imagination-land content, because the resolver checks every symbol, hook, and route against the real codebase. They will still try. It just doesn't get through.

That closes the loop. Humans and agents read Atlas. Agents update Atlas. Atlas gates those updates with the same static checks. The map stays honest because nothing writes to it unchecked.

What it changed

Exploring a new feature went from days to hours, once agents read Atlas first. Bug fixes went from days of guessing to inside a day, with real confidence the fix holds, because the map shows every part the change touches. And the hallucinations, the invented hooks and symbols agents love to ship, atlas lint stops at the gate.

What's next

Atlas already proves every documented surface against the code. But this isn't the end of the story. Of course I still have some ideas to improve it.

I would want it to hand the agent exactly the context it needs at the point of work: the rules, the contracts on the surfaces being touched, the active collisions, and the historic footguns tied to those files.

Beyond that I can imagine: a staleness engine for docs lagging behind code, an auto write-back that proposes new public surfaces from a diff, a living package map that regenerates itself, a PR review bot with inline annotations.

The rails

Writing a plan before the code is real overhead. The trade is one plan now to skip a week of merge hell later. It pays off every time.

We aren't the first to use LLMs and AI agents. The use of agents stopped being a differentiator the week they saw the light of day. The vanguard part is the rails. We built them before we let the agents drive: a stack that knows itself, in a language a machine can read without guessing, with tooling that proves the map still matches the territory on every commit.

Agents do the heavy lifting. Humans review and approve every change. Atlas leaves the coding to humans and agents. Its job is keeping that code aligned with the architecture we already agreed on.

Fewer collisions. Less drift. A stack that stays coherent as it grows.

Turns out we knew what we were doing. And for once, so did I.