Ronin Loop · Clief Notes

Ronin Loop

My agents were lying to me and I couldn't tell

Retired NYPD detective. I run three businesses on Codex/Claude Code agents across three Macs. Last week I found out my monitoring layer had been reporting healthy while doing nothing, for months.

Not broken. Lying. Different problem, and it's the one nobody talks about.

Here's what a day of tearing it apart taught me, and the loop I built out of it.

The failure nobody names

Nine distinct ways my system reported success while failing. A few:

- A wrapped job that exits 0 having done nothing gets reported GREEN. ~135 jobs affected.

- A missing receipt is indistinguishable from a successful run. A job that never ran looks identical to one that passed.

- Drop any `.json` file into a receipts directory and you fabricate a clean bill of health for 39 states. Worse — a stray file with a newer timestamp *displaces* a real failure record. I had a genuine timeout that got overwritten and turned green.

- My alerting worked perfectly. It delivered six RED alerts over six days. Nobody read them.

That last one matters most. The system wasn't blind. It was ignored. Alert fatigue is a design failure, not a discipline failure.

Meanwhile a detector I'd built to catch exactly this had been running for ten days, correctly identifying that no leads had shipped to my ops manager — and telling nobody, because delivery was never armed. A working alarm wired to nothing.

---

The loop

Every agentic framework I read describes the same thing: an agent that generates, checks, and repeats. What none of them specify is what happens when the check says no.

That's where mine lives.

```

ROUND N:

1. CONFIRM reproduce the defect behaviorally, never fix on description

2. FIX smallest diff at root cause, harden against the class

3. SELF-TEST builder proves it. Verdict here is PROVISIONAL. Always.

4. REGRESSION re-run every prior test, PLUS generate adversarial inputs

against the surface this round just changed

5. AUDIT hand to a different model as an OPEN QUESTION —

"present or absent?" never "confirm this is fixed"

6. VERDICT PASS closes it. FAIL classifies the failure, re-prompts

itself with the finding, opens round N+1.

```

Three hard stops, not one:

- **Round cap: 4.** Measure twice, cut once — three times if the cut is expensive.

- **No-progress:** same blocker twice, halt immediately. If two measurements return the same wrong number, the problem is the tape, not the measurement.

- **Budget:** declare a ceiling upfront, checkpoint and halt on crossing it.

And the exit condition that makes it work: **an item is done only when an auditor that COULD have returned FAIL returned PASS.** The builder never closes its own item. Ever.

---

## What it caught

Round cap fired twice. Both times it was right.

One item took three rounds patching field by field — each fix reasonable, each surfacing a new version of the same problem. Round 4 stepped back and asked what shape the problem actually was, and closed the whole class in one change. **The slower path was shorter.**

No-progress fired once. Two fixes, one root cause, halted at round 2 instead of spending round 3 on a third symptom while a fourth waited.

The adversarial pass — generate new inputs against what you just changed — caught the receipt-forgery hole. A 22-test regression suite walked straight past it. Only a constructed input found it, because prior-round tests are blind to a defect this round introduced. They were written before the change existed.

---

The one thing I haven't seen anywhere else

Every framework says split the maker from the checker. None of them say what happens **when the checker is the thing being repaired.**

I hit this live. One of the defects was that the review mechanism accepted any executable as a valid reviewer — someone wrote a 4-line shell script that just printed "PASS" and it was accepted as a legitimate audit.

So: who audits the fix to the auditor?

Not the auditor. That's maximally circular. Its trustworthiness is the thing under repair.

The rule I wrote: **when a defect concerns the review mechanism itself, the reviewer may help write the fix but cannot validate it.** Validation has to be direct behavioral proof — recreate the exploit, confirm it's now rejected.

---

Being honest about what didn't work

I tried to make the loop enforce itself instead of asking nicely. Built a gate that would block any commit without an audit receipt. Ran it through the loop.

It failed its own audit. Four ways:

- It failed open, not closed. Only one exit code blocked; any other error let the commit through. A typo in the gate file would silently switch the whole thing off.

- The property I was proudest of — "only a specific event can write an audit receipt, so the auditor can't be the builder" — was half true. That event fires for every subagent. Any subagent doing any task became an auditor of record.

- The bypass surface was a sieve. `git commit -m 'fix; tidy up'` got through, because the parser split on separators before tokenizing.

- Installing it would have deleted an unrelated safety hook that blocks outbound sends.

Root cause, and both reviewers reached it independently: a hook reads a string and has to infer what the shell will do with it — and that meaning lives in aliases, functions, PATH, and variables the string doesn't contain. That false-negative class is unbounded. No amount of parsing closes it.

The check has to move to where the operation actually happens, after expansion. Different build.

I'd rather post that than pretend it worked.

---

What I'd tell you to steal

If you take one thing: generation is cheap now, verification isn't. Your loop only spins as fast as its checking half. Most people are optimizing the wrong side.

If you take two: a green test suite is evidence about everything except the change you just made.** Those tests were written before it existed.

And the one that cost me the most to learn — knowing a trap by name doesn't stop you falling in it. One of my panels logged a specific failure mode to its own reference doc, then committed that exact mistake an hour later. Warnings don't work. Mechanical checks do.

---

Honest limits: one operator, one system, one very hard day. This isn't validated across a fleet. It's a design with receipts, which is a different claim.

Happy to go deeper on any piece if it's useful.

1 comment

Ronin Loop