Keep a coding agent climbing until a jury of judge agents signs off

Mokhles El Heni spent two weeks letting judge agents, not the worker, decide when work on their app was done: the UI rubric went from 65 to 80 of 80 and server start fell from 1,505 ms to 135 ms.

Done withClaude CodeCodex· Nov 2, 2026

What they did

Mokhles El Heni, who builds wisp, an open-source app for running coding agents in the background, tested a rule on wisp itself for two weeks: the agent doing the work never gets to say it is finished. Each task got a goal a judge could check, such as every row of a scorecard at 9.5 or higher, or server start under 200 ms with 770 tasks. For the UI run the first instruction was to write a measurable 16-row scorecard (distinct font sizes per screen, axe-core violations, controls under 44px, values shown twice) before changing anything, measured in 20 views across widths, touch, phone and both themes. After each round, up to five judges (Claude Code, Codex, and a long-lived UI reviewer agent) read the goal, diff, test results and screenshots, with the worker's own report passed in last and marked as a claim. They voted under an All, Majority or Any rule. Their notes on what was still missing became the next round's prompt, with no person in between. The loop stopped when the goal was met, judged impossible, or stalled for two rounds, not after a fixed round count.

How it went

The UI run took four judged rounds to go from 65 to 80 of 80, ending with 3 to 5 font sizes, zero axe-core violations and no small phone controls. The performance run cut server start from 1,505 ms to 135 ms, desktop open from 3.7 s to 0.95 s, first phone load on throttled 3G from 60.1 s to 18.2 s, and memory from 155 to 33 MB. That goal, phrased as nothing left to improve, was ruled impossible by the judges after two rounds, so it ended at about 85% instead of looping. A stricter fresh judge later knocked the saturated UI score back to 74.

Worth knowing

The author makes the tool the runs went through, so this is a builder describing their own product on their own app. A vague goal still produces work but can never be met; state it as a number. Once a rubric is maxed out it stops telling you much, so the author swapped in a stricter judge and a bigger rubric. The post does not say which agent did the coding work.

Try it

With Claude Code· 5 parts to fill in

Work on [my app] toward this goal: [measurable goal, e.g. every scorecard row scores 9.5 or higher, or server start under 200 ms on my real data]. If there is no scorecard yet, write one first where every row is a number taken from a screenshot, profiler or test run, then make changes. After each round, collect evidence that does not depend on your opinion: test results, the diff, [accessibility or perf scripts], screenshots at [widths]. Hand that evidence to [judge agents] who score it independently; your own summary goes last and counts only as a claim. Use their list of what is still missing as your next step. Stop and report when the judges say the goal is met, when they say it cannot be met as written, or when two rounds in a row change nothing, and ask me before changing the goal.

Square brackets are the parts to fill in. The prompt is ours; the story is theirs.

Where they said it

Related