What they did
jidonglab builds Refmade, a desktop app that wraps the Claude Code and Codex command-line tools, and has those agents write most of the code from specs while they plan and review. On September 29 they removed a rule that refused team mode unless Claude Code was signed in with a plan login, so API-key and other sign-ins would work too. The next day they ran an adversarial review: a few agents told only to find how the change fails. After the first repair, they ran a second review pass with one agent aimed at the repair itself. They reproduced each finding with a fake API key and a local test server standing in for the API.
How it went
The first pass found that the readiness check read the user settings file but the restricted team run did not, so some users saw 'ready' and then got a failed run or a silent switch to another stored Claude account. The repair passed that settings file back to the run, and the second pass found the file could also widen folder access beyond the one the user picked. The final fix passes it only when the sign-in comes from it. The same review also caught Codex being forced to OpenAI for Azure users and API keys leaking between the two tools. Both regressions were fixed before the October 1 release, with about 1,100 tests and mutation checks passing.
Worth knowing
Both bugs came from the repairs, not the original code, which is why the author reviewed the fix as well as the change. The passing tests only show the code matches the specs; the author says no usability testing with real users was done. The post does not say exactly which agent ran each review pass.