July 9, 2026 · Marcus Webb, The Beauty of Mathematics~8 min read
The job used to have a name, and the name kept moving. First it was prompt engineering: you fussed over the wording of one request. Then, around the middle of last year, Andrej Karpathy pointed out the real unit had grown — it wasn't the prompt anymore, it was the whole context you handed the thing: the system message, the docs you pulled in, the tool list, the CLAUDE.md sitting in the repo. He called it context engineering, and it stuck. Now, June of this year, Addy Osmani put a word on the next step and it was everywhere inside a couple of weeks: loop engineering. The unit is the loop the machine runs — generate, check, steer, retry, stop. And underneath all three names, there's one line everybody keeps repeating, because it's the whole game: the verifier is the bottleneck now, not the model.
30-second read
The unit of engineering kept climbing — the prompt (2023), the whole context (2025), now the running loop (2026). Under all three sits one line: the verifier, not the model, is the bottleneck.
There are two kinds of checker and they are not interchangeable: a deterministic check runs the code and returns the same verdict every time; a model-graded check asks another model "is this good?"
The trap: a model-graded judge is the same probabilistic component the loop was built to supervise — so a loop can stop, confidently, on the wrong iteration.
1The checker
The thing you're actually building is the checker
Run an agent in a loop and something has to decide: good enough, or go again. That something turns out to be the hard part.
Here's the thing. When you run an agent in a loop, the model spits out a candidate. Something has to look at that candidate and decide: good enough, stop — or nope, go again. That something is the verifier. And the plain fact nobody wants to sit with is that the verifier is the hard part. Getting the machine to produce another draft is cheap. Deciding when a draft is done, and being right about it, is the whole job. The prompt is one node in the loop. The context is the state you carry between laps. But the piece that determines whether the loop ever stops on something good — that's the checker, and that's what you're really engineering, whether you meant to or not.
⚡
Why it matters: the generator got cheap; the whole leverage moved to the thing that decides when to stop — the quiet, unglamorous checker nobody wants to build.
This is where an old book about search engines turns out to be about your loop. Wu Jun, in The Beauty of Mathematics, keeps hammering one point: a system's ceiling is set by the quality of the yardstick it optimizes against, not by how flashy the engine is. A search engine isn't as good as its crawler is fast; it's as good as its relevance metric is honest. Get the metric wrong and a faster crawler just gets you to the wrong answer sooner. Same wrench, different bolt. Your loop is only as good as the verifier that grades it. The generator is the crawler. The verifier is the metric. Pour compute into the generator and skimp on the checker, and you've built a very fast way to confidently finish the wrong thing.
Put it another way
The generator is the crawler; the verifier is the relevance metric. A faster crawler pointed at the wrong metric just reaches the wrong answer sooner — same wrench, wrong bolt.
The unit of engineering climbed from the prompt (2023) to the whole context (2025) to the running loop (2026: generate→check→steer→retry→stop). Inside the loop the checker forks: a deterministic check runs the code and returns the same verdict for the same input, while a model-graded check asks another model "is this good?" and inherits the exact unreliability the loop was built to contain — a probabilistic judge deciding when the probabilistic generator is done. Source: Addy Osmani / Peter Steinberger on loop engineering, June 2026; DEV Community & ADTmag, July 2026. Framing: Wu Jun, The Beauty of Mathematics — a system's ceiling is set by its evaluation metric, not its generator. A conceptual map of the discourse, not a benchmark; one engineer's read.
2Two kinds of check
Two kinds of check, and they are not the same tool
There are two ways to build a verifier, and people swap them like they're the same tool. They are not.
Now the part that actually matters on the bench. There are two ways to build a verifier, and people mix them up like they're interchangeable. They're not. The first is a deterministic check: run the code, look at the exit status, scan for a forbidden import. Same input, same verdict, every single time, no opinion involved. It's a feeler gauge — the gap is 0.010 or it isn't, and the gauge doesn't have a mood. The second is a model-graded check: you hand the candidate to another model and ask, is this good? Is this explanation clear? Does this read as rude? Those are real questions a deterministic check can't touch, because you can't write an assertion for "clear."
Deterministic check
Run the code, read the exit status, scan for a banned import. Same input, same verdict, no mood — a feeler gauge.
Model-graded check
Ask another model "is this clear?" It reaches what you can't assert — but inherits the same unreliability the loop was built to contain.
So the model-graded check reaches things the deterministic one never will. Fine. But look at what you just did. You put a probabilistic judge in charge of deciding when your probabilistic generator is finished. The judge is the same kind of machine you built the loop to babysit in the first place. It's as if you caught the fox at the henhouse, so you hired a second fox to keep an eye on the first one. This is Wu Jun's other split, the deterministic-versus-statistical line, landing right in your CI pipeline: the deterministic signal is dumber but it doesn't lie to you; the statistical one is smarter but it drifts, exactly where you need it not to. A loop with a model judge can stop, and stop confident, on the wrong iteration — and nothing in the loop will tell you it did.
The catch
You hired a second fox to watch the first fox. A model-graded verifier reaches "is this clear?" that no assertion can — but it's the same probabilistic component the loop exists to supervise, so it can call the wrong draft done, and sound sure while it does it.
3Thin context
Most of your context isn't even doing anything
The state you carry between laps is thinner than it looks.
And it gets worse before it gets better, because the state you're carrying between laps is thinner than it looks. Somebody ran a deterministic analyzer — the honest kind, the kind that doesn't have a mood — over 28,721 repositories and counted what's actually in those instruction files everybody's grooming. The median file had 50 content items in it. Twelve of them were actual directives. The other 38 were headings and structure and vibes the model is free to ignore. So most of the "context" you carefully assembled isn't coupled to behavior at all. It's furniture. You measured it as if it were doing work, and mostly it was just sitting there, which is exactly the trap Wu Jun warns about — you can't manage what you've mismeasured, and a proxy you can't trust is worse than one you know is crude.
Actual directives
12 / 50
Headings & vibes
38 / 50
Median instruction file across 28,721 repositories: 50 content items, only 12 actual directives (State of AI Instruction Quality).
4This week
What this means for you this week
Concretely, here's the move.
If you're running agents in a loop, go find your verifier and ask one blunt question: is it a feeler gauge or a second opinion? For anything you can pin down — does it compile, do the tests pass, is that import banned — use the deterministic check and stop apologizing for how boring it is. Boring is the point. Boring doesn't drift. Save the model-graded judge for the stuff you genuinely can't assert, know that you've hired a fox to watch a fox, and don't let it be the only gate before something ships. The generator got cheap. Everybody's staring at it because it's the flashy part. The whole leverage moved to the thing deciding when to stop — the quiet, unglamorous checker nobody wants to build. That's the metric now. Build that, or your loop just gets to the wrong answer faster.
Discourse from Addy Osmani and Peter Steinberger on loop engineering (June 2026), synthesizing Boris Cherny; reporting via DEV Community, "Prompt Engineering, Context Engineering, Loop Engineering: What Actually Changed" (2026-07-08), ADTmag, "Loop Engineering Emerges as Developers Put AI Coding Agents on Repeat" (2026-07-01), and civic.io, "Context is King" (2026-07-01). Context-file figure from the "State of AI Instruction Quality" analysis: a deterministic analyzer over 28,721 repositories found a median of 50 content items but 12 actual directives per instruction file. The "unit kept climbing" framing (prompt → context → loop) traces to Andrej Karpathy's mid-2025 naming of context engineering. Framing from Wu Jun, The Beauty of Mathematics (a system's ceiling is set by its evaluation metric, not its generator; the deterministic-vs-statistical split). Honest limits: this is a fast-moving field and one working engineer's read of an emerging term, not a benchmark or a verdict on any specific tool.
所以模型评判这条，能够到确定性那条永远够不着的东西。行。可你回头看看你刚干了啥。你请了一个概率性的裁判，让它来判你那个概率性的生成器什么时候算完事。这个裁判，跟你当初建循环要看住的，本就是同一种机器。这就好比你逮着黄鼠狼在鸡窝口，于是你又雇了第二只黄鼠狼，去盯着第一只。这正是吴军另一条分野——确定性对统计——直接落进了你的 CI 流水线：确定性那路笨，可它不糊弄你；统计那路聪明，可它会漂，偏偏漂在你最不想它漂的地方。带模型裁判的循环，会停，而且理直气壮地停在错的那一轮——循环里没有一样东西，会告诉你它停错了。
讨论来自 Addy Osmani 与 Peter Steinberger 论 loop engineering（2026 年 6 月，综合 Boris Cherny）；报道见 DEV Community "Prompt Engineering, Context Engineering, Loop Engineering: What Actually Changed"（2026-07-08）、ADTmag "Loop Engineering Emerges as Developers Put AI Coding Agents on Repeat"（2026-07-01）、civic.io "Context is King"（2026-07-01）。上下文文件数据出自 "State of AI Instruction Quality" 分析：一个确定性分析器扫 28,721 个仓库，发现指令文件中位数是 50 条内容、其中仅 12 条为真正的指令。"单位一路往上爬"（提示词 → 上下文 → 循环）的说法可溯至 Karpathy 2025 年年中对上下文工程的命名。框架取自吴军《数学之美》（系统的上限由评价指标定，不由生成器定；确定性与统计之分）。诚实的边界：这是个快速演进的领域，也只是一位一线工程师对一个新词的解读，不是基准测试，也不是给哪个具体工具下判决。
だからモデル評価のほうは、決定的なほうが永遠に届かないところに届く。いいでしょう。でも、いま自分が何をしたか見てください。確率的な審判を雇って、確率的な生成器がいつ終わるかを、そいつに決めさせてる。その審判は、そもそもループで見張るために作った、あの機械と同じ種類なんです。鶏小屋の入り口でイタチを捕まえたので、その一匹目を見張らせるために、二匹目のイタチを雇った——みたいな話。これが呉軍のもう一つの分かれ道、決定的 対 統計、が、そのままあなたの CI パイプラインに落ちてくる。決定的なほうは鈍いけど、ごまかさない。統計のほうは賢いけど、ずれる。よりによって、ずれてほしくないところで。モデル審判付きのループは、止まる。しかも間違った周回で、堂々と止まる。そしてループの中の何一つ、それが間違いだと教えてくれない。