Two engineering leaders, in two separate first sessions, asked me the same question in the same words: "How do you trust the AI output?"
I don't.
That's not a dodge. I wrote earlier this year that the most important question in software engineering in 2026 is "how are you verifying AI's work?" and I still think that's the whole game. But when I answered these two, I noticed something. The first two things everyone reaches for are exactly the two things that stop working once agents write most of your code.
Unit tests don't catch customer regressions
Agents love unit tests. Ask one to "add tests" and you'll get forty of them in a minute, all green, all testing that the code does what the code does.
A unit test is the implementation's opinion of itself. It pins down a function's understanding of its own contract. That's useful while the agent is writing, as a fast inner loop, and for the handful of places where the logic really is the product, like rounding money. It is not where trust comes from.
Customers don't file bugs against functions. They file bugs against behaviors: the page that stopped loading, the message that arrived twice, the login that works in Chrome and not on the phone. Those regressions live in the seams between components, in the runtime, in the browser, in the wire. A unit suite can be 100% green while every one of them ships.
And there's a second problem specific to agents. The agent that wrote the code also wrote the tests, from the same misunderstanding. If it misread the requirement, the tests encode the misreading and pass. With a person at the keyboard, someone is holding the requirement steady while the tests get written. With an agent, the same model sits on both sides of the assertion.
Keep your unit tests. They're fine. Just stop counting them as evidence.
Manual code review doesn't scale
The other reflex is "we'll just review everything carefully." Then, a week later, "we'll review the risky ones."
I've written before that manually reviewing AI code the way we review human code doesn't scale. The nuance I'd add now is that it was never about how many diffs you read. Any diff a human has to read before it can move is a diff waiting on a human, and that wait is the bottleneck.
Volume makes it obvious. Ten agents produce more pull requests than a team can read, and the quality sits everywhere on the spectrum from "merge as is" to "why is this wasting my time." But cutting the volume doesn't fix it. Read only the risky ones and you've added a triage step that's also a human. The other ninety percent now merges with nobody having looked at all, and the ten percent still queues behind a person. Read a sample and you have a sample. Either way, the work that gets read waits, and the work that doesn't get read is trusted on nothing.
The less obvious part is what it does to the reviewer. Reading your fortieth agent diff of the day, you stop reading and start pattern matching. You become an approval button that occasionally says no.
So the question isn't which diffs a human reads. It's what has to be true about a change for it to move without one.
Verification is a cycle
The principle behind everything that follows: every layer of trust has to live outside the code the agent wrote, and as close to the customer as you can afford. The agent can rewrite its own tests. It can't rewrite the reviewer, the browser, the wire, the second implementation, the lower environment, or production telemetry.
And the layers aren't a wall a change climbs once. A change gets reviewed, runs through CI, lands on a lower environment, reaches the sandbox, goes out as a canary, and what production says about it comes back as an alert that lands on an agent, which opens the next change. Eight steps, in the order a change moves through them.
1. Agent reviews you can trust
Review doesn't go away. It moves. Review moves from a human reading a diff to agents reviewing under rules, and the rules are what you audit.
The shape that works: the agent that writes the code is the driver. It can commit and open a pull request. It cannot merge. A separate reviewer agent reads the change and gives correctness advice the driver can take, ticket, or ignore. A separate security reviewer's red is a veto. Both sign what they reviewed, bound to the exact commit, so a new commit is a new review. Merging is done by a machine that acts only when CI is green on that exact commit and the signatures are present on it. "Ready to merge" is something an agent reports, never something it does.
If you're in a regulated shop, you still need a named person on the change, and that's fine. The person stops reading diffs and starts signing off on evidence. The signed reviews, the outer-spec run, the canary plan. That's a smaller, auditable job than line-by-line reading, and at agent volume it's the only version that survives an audit.
The machinery underneath, the signing keys, the policy, the promotion rules, is its own topic and I wrote it up separately in the control-planes note. For this question it's enough to say this. A review is no longer a person's opinion. It's a signed claim about exact bytes, made by a role that was allowed to make it.
2. Outer specs
An outer spec is a test that drives the system through a surface a customer would actually use: an HTTP API, a browser, a terminal. It never imports the implementation. It doesn't know which language the server is written in. It boots the real thing, waits for it to answer, and then behaves like a user.
For an API, that's contract tests over the wire. For a web product, that's browser tests, Playwright or whatever you prefer, clicking real buttons on a real build. For a CLI or anything an agent operates from a terminal, that's a tmux session with something typing into it and reading what comes back.
The point of an outer spec is that the agent can't satisfy it by changing what the test means. It has to change what the system does. That single property is worth more than any number of unit tests, and it's the first thing I'd ask a team to build before letting agents loose on a codebase.
Outer specs are slower and more annoying to write than unit tests. That used to be a reason not to write them. Agents write them now, so the excuse is gone.
Two habits keep the suite honest. Every acceptance item names a test that runs in CI, never a demonstration. A demo runs once and protects nothing after, so if you catch yourself asking an agent to "show me it works," ask for the test name instead. And every test that claims a defect is absent gets shown failing with the defect present. An assertion nobody has watched fail is decoration. When an agent reports a test written, the reply is "show me it failing with the bug put back in."
3. Two implementations of one spec
This is the one people push back on, so let me define it before defending it.
Build the backend twice. Two languages, two squads of agents (a squad being a few agents working one change together, with a written brief), one set of outer specs that both have to pass. Deploy one. Keep the other alive in CI as the second opinion.
Why pay that tax? Because of the oracle problem. With a single implementation, that implementation silently becomes the spec for anything the written spec doesn't pin down. Every ambiguity gets resolved by whatever the code happens to do, and the tests learn to expect it. With two, an ambiguity shows up as a disagreement. Sometimes one side has a bug. Sometimes the spec never said. Both are worth finding, and neither would have surfaced with one backend, because there was nothing to disagree with. Budget for the triage, and hold the spec, never either implementation, as the thing that decides.
This also happens to be the thing agents are absurdly good at. Maintaining two backends with a human team is cruel. With a squad, keeping the second one green against a spec change is a brief you send in the morning. The economics that made this a ridiculous idea in 2023 are gone.
4. Spend a hundred times more on CI
Not ten percent more. Not double. If you have comprehensive outer specs, you should probably be spending two orders of magnitude more on CI than you do today. If CI is a rounding error next to your inference bill, that ratio is the bug.
The outer specs only buy trust if they run on every change, quickly enough that nobody is tempted to skip them. A highly leveraged team, a handful of engineers each running several squads, produces changes at a rate no human process can absorb. The only thing standing between that team and a production incident is a comprehensive outer suite that runs fast. So the suite has to be comprehensive, which makes it big, and it has to be fast, which makes it expensive. There is no cheap version of that combination.
Give CI a budget in minutes. Under five is the target I'd hold a squad to, and you get there with shard count and hardware, not by making the suite smaller: self-hosted runners on bare metal, aggressive caching, a dedicated Mac if your platform needs one. Treat CI capacity the way you treat inference capacity, because it's the same bottleneck wearing a different hat.
A 45-minute pipeline turns every agent into a very expensive way to wait. And a slow pipeline is where "we'll just run the outer specs on every PR" quietly dies, which puts you right back at trusting the output on nothing.
5. Lower environments with continuous deployment, and let the agents be the traffic
Every merge that passes CI should deploy to a lower environment (dev, staging, whatever you call the rungs before production) automatically. No approvals, no tickets, no human in the loop. If that sentence scares you, that's the fear to work on, because the alternative is an environment nobody deploys to and therefore nobody trusts.
Then put traffic on it. Mirror production requests against the candidate if you can, with the candidate's writes and outbound side effects stubbed, and only with data that's allowed to exist in a lower environment. And if your product is one that agents can use, point your own squads at it. Let the agents be the traffic. Make them the users. When it breaks, the agents that broke it are the first to notice, because their own work stops flowing. That is the cheapest, most honest load test you'll ever run.
6. A sandbox your customers actually use
Most companies with an API have a sandbox: the environment customers integrate against before they go live. It usually runs whatever shipped last month, and its whole job is to keep customers from breaking production while they build.
Flip it. Put the candidate build in the sandbox first, and let your customers' integrations be a verification layer you never had to write. Every customer that integrates with you has their own tests, their own traffic shapes, their own strange edge cases you never thought of. When the candidate breaks one of them in the sandbox, that's a regression caught before production, by the exact customer it would have hurt, for the price of one customer's afternoon instead of everyone's outage. Tell them the sandbox runs the candidate, and fix what they find fast. Otherwise you're outsourcing QA to people who pay you.
On naming: staging is yours, the sandbox is theirs. Staging is where your own team and your own agents hit the candidate, which is the previous step. The sandbox is where the outside world does. You want both, and the sandbox is the more honest of the two, because you don't control what runs against it.
One warning. This only works if the sandbox is literally the product. The moment it grows its own login, its own data model, or a special SDK "for testing," your customers are integrating against something you don't ship, and whatever they catch there tells you nothing about production. Run the same code and hand out the same SDK. The only thing that should be fake is the money.
7. Wire observability to the agents, not to you
Most teams have telemetry. Fewer have telemetry an agent can act on.
Two changes. First, a surface doesn't ship without its telemetry in the same change. Spans, metrics, and the alert that reads them all land together, or none of it lands. Second: every alert names one actor and one action, and the actor is an agent. The page goes to the agent that owns the fix, as a message it can read, with the runbook step attached. A human sees the same alert. The human is not the router.
Then add the probe that fixtures can't replace. After every deploy, and on a timer regardless, do the thing you're most afraid of breaking, for real: a signup through the real payment provider, a login through the real identity provider, an order all the way to the confirmation email. The only test of "can we still do X" is doing X.
8. Canary deploys with easy rollbacks
The last net, and for a product with customers the one I'd never skip.
Ship to a slice. Watch the telemetry from step seven for a few minutes. If anything moves the wrong way, roll back. Rolling back is way faster than rolling forward. An agent fixing forward has to understand the failure, write the change, get it reviewed, get it through CI, get it promoted, get it deployed. Even with everything in this note running well, that's several minutes, and you don't want to sit through it several times a month with customers on the wrong side of it. A rollback is a feature flag flipping the canary off, or loading the previous production setup. Seconds.
The catch is that rollback is only a switch if every change is backward compatible with the build before it. Expand-then-contract migrations instead of one-shot schema changes. Additive events, never renamed ones. Flags instead of branching deploys. That's the expensive part, and it's the work that makes "roll back, don't fix forward" true instead of a slogan.
So build rollback as a switch you can flip while still reading the alert. If rollback requires a meeting, you don't have rollback, you have a postmortem scheduled in advance.
This is also where "fix forward" stops being a virtue. Fix forward is fine when the blast radius is a lower environment and the agents have time. In production, the fastest fix is a feature flag toggle or the previous build, and the agent's job is to open the next change after the customers are already safe. That alert becomes the brief for the next change, and the next change goes back to step one.
What I'm doing about it
I'm building this cycle in roughly the order above, on my own work, and none of it is finished. The order matters more than the completeness. Review rules and outer specs first, because everything else stands on them. CI speed second, because without it nobody runs the outer specs. The rest arrives when the volume forces it, and it will.
Trust isn't something you extend to the model. It's a property of the system around the model, and you build it by pushing verification outward until the agent can't satisfy a check by changing what the check means.
Start with the outer specs. Everything else in this note is a way to run them more often.