Check every claim in a draft post against git with an auditor agent

Sungsoo Youn's separate auditor agent caught three wrong sentences and two unreproducible numbers in drafts about his Claude Code setup, and it took three passes to return PASS.

Done withClaude Code· Oct 22, 2026

What they did

Youn runs an autonomous Claude Code agent every day on one Windows PC, guided by a CLAUDE.md and memory files it reads before it touches anything. He writes field notes about that setup using numbers taken from the agent's own ledgers and from git. Before anything goes out, a separate auditor agent checks the drafts and returns PASS or FAIL. In this round it checked a field note and two short posts queued for other channels, one of them in Korean, all counting commits and shell calls over a stretch when a handoff note sat above 400 lines.

How it went

The counts were right but the window was not: three sentences opened it with a date, while the count actually began at a commit at 1:36 PM. Counting from midnight gives 90 commits and 104 shell calls instead of 84 and 96, and six of those commits left the note under 400 lines. A test he had written asserted the wrong wording, locking the mistake in. The auditor also flagged two figures, in a progress log and a code comment, that it could not rebuild, so he deleted both and wrote down its recount with the counting rule. The second pass failed on the same error in two more files. The third passed.

Worth knowing

The audit works because it recounts from the source data instead of trusting the draft, and because a test that copies a sentence copies its error. After a fix, look for siblings: the second pass failed only on copies he had not checked. A total without its counting rule cannot be checked at all. The author sells a book about this setup.

Try it

With Claude Code· 3 parts to fill in

You are an auditor. Read [the draft post or report] and list every factual claim and number in it. For each one, rebuild it yourself from [the git log, session logs or data files], stating the exact counting rule and start point you used. Mark each claim PASS or FAIL, and for every FAIL give the figure you got and what the sentence should say instead. Flag any number you cannot reproduce. Then search [the repo] for other copies of each failed wording. Do not edit anything. End with one verdict: PASS only if every claim reproduces.

Square brackets are the parts to fill in. The prompt is ours; the story is theirs.

Where they said it

Related