Your Prompt Has No Way to Notice When It Becomes Wrong

Last week half the instructions in a lot of people’s setups expired. Anthropic shipped Opus 5 with a prompting guide that is substantially a list of things to remove — the verification steps, the “double-check your answer,” the conservatism cap on review prompts that now makes the model report fewer real bugs.

That post answered what to delete. It left the more uncomfortable question alone, and it is the one worth an essay:

Why didn’t anything tell you?

Not “why didn’t Anthropic tell you” — they published a document on the day. Why didn’t your system tell you? You have tests that fail when an assumption breaks. You have types that refuse when a shape changes. You have CI that goes red. A premise inside your own setup stopped being true, and every one of those mechanisms stayed green, because none of them was watching the place where the premise lived.

The instruction had no way to notice

Here is the shape of it, stripped down.

You once added a line to a prompt: verify your work before responding. You added it for a good reason — a model shipped you an answer that didn’t survive a second look, and the line fixed it. It was a real fix for a real failure.

That line encodes an assumption: this model does not reliably check itself. The assumption was true when you wrote it. It is not true now.

But the line doesn’t know that. It cannot know that. It is prose — text the model re-reads and re-interprets on every single run, with no connection to the fact it depends on. There is no assertion, no type, no gate. When the assumption underneath it died, the line kept executing exactly as written, and everything downstream stayed green.

That is not a prompt-quality problem. Rewriting the line more carefully doesn’t help; it was already precise. Adding examples doesn’t help. The problem is where the check lives.

Premises break in two directions, and one is much worse

I keep a framework for this — The Control-Plane Placement Framework — and the part that earns its keep is the failure asymmetry.

Downward is the familiar direction. A cleanup routine I ran had a liveness check written as prose: read the recorded process ID, see whether a process with that ID is running, and if not, treat the work as abandoned and close it out. Every clause correct. Then a process restarted underneath a live task — the task continued, but the ID on record now named something that no longer existed. The check did exactly what it said. It closed live work, confidently, on an assumption that had quietly stopped holding.

Painful, but loud in hindsight: something visibly wrong happened, and it could be traced.

Upward is the direction Opus 5 introduced, and it is worse. Here the premise breaks because the model got better. Your “double-check your answer” now runs a verification pass on top of one the model already performs. The output is correct. The tests pass. The review looks fine.

There is no incident. Nothing to trace. The only evidence is a bill nobody audits — and, in the case of the conservatism cap, a finding that simply never arrives. A bug the model located, considered, and declined to mention because you asked it to be conservative.

A premise break that produces correct output defeats every quality gate you own, by construction. Your gates measure whether the answer is right. This failure produces right answers. That’s the whole problem.

Two places anything can live

The framework’s premise is simple enough to carry in your head: every element of how your agent work runs — which step goes next, what’s true right now, whether a step succeeded, what happens on failure, when a human must approve — lives in exactly one of two places.

In prose: instructions the model reads and is trusted to follow. Fast to write, endlessly flexible, and re-derived from scratch every run. The same input can route two different ways on two different days. A broken assumption is followed faithfully. And the run can’t be replayed, because there was never a deterministic sequence to replay — only a transcript, which is a trace, not a state.

In a control plane: code the model runs inside but cannot reinterpret. Slower to build, rigid on purpose. Identical every run. Refuses the illegal transition outright. Replays exactly.

The instinct is to read that as a quality ranking. It isn’t. It’s a failure-mode choice, and the decision is per-element, not all-or-nothing — the reasoning inside a step belongs in the model, always. Hard-code the thinking and you’ve rebuilt a rules engine and thrown away the reason to use an agent. What belongs in the plane is the routing between steps.

Two elements make the point without needing the full set.

Human gates. “Check with a human before anything irreversible” in a prompt is a request. The model can interpret its way past it under load — and will, eventually, on the run where it matters. The same gate as a blocking step in the plane has no edge around it. One is an instruction; the other is an enforcement. That distinction is the entire difference between a gate you have and a gate you believe you have.

Shared state. “Remember the decision you made earlier” delegates truth to the transcript. Held in the plane, there’s one authoritative record of what’s true — and a run you can actually replay rather than merely re-read.

Get an element on the wrong side and the system is flexible right up until the moment a silent premise change routes it somewhere nobody can reproduce.

The fix that isn’t a fix

The obvious remedy, when you see all this, is a resolution: review your prompts whenever a model ships.

Look closely at what that is. It’s prose. It’s a procedure a human is trusted to remember, whose own premise — that somebody noticed the release — breaks exactly as silently as the one it was meant to catch. You’ve patched a prose failure with more prose and added a step that will decay the first busy week you have.

The placement move is different. Make the model version part of the state your system actually carries, and make a version change a step that blocks until the instruction set and effort settings have been re-swept. The version stops being an ambient fact your prompts quietly assume and becomes a value the system holds and checks — which is the same fix as replacing that stale process ID with a heartbeat the running process refreshes itself. Different stale fact, identical remedy.

Then, when a launch falsifies your compensations, the thing that notices is not your memory. It’s a step that won’t proceed.

The point

The reason your instructions went stale without a sound is not that you were careless. You wrote them for real failures and they worked. It’s that you wrote them somewhere that has no way to notice when it becomes wrong.

Prose can’t fail loudly. That’s not a flaw in your prompt — it’s the nature of the place you put it. And every safeguard you’re proudest of is sitting in that place right now, still running, still confident, guarding against something that may have stopped happening months ago.

Move the checks that must never be wrong somewhere that can refuse. Leave the thinking in the model.

If you want the version of this that goes deeper — where each element belongs, and how to audit which side yours are on — that’s the software-engineer track.