What they did
Deshpande's team builds a logistics platform for warehouse and yard operators, and one customer wanted to ask plain questions about about 118 reports from a chat client. They wrote two MCP servers on Azure Functions, one with 7 tools to find, run and read reports and one with 19 lookups that turn names like a depot into ids, with a Claude Sonnet 5.5 agent on top. Real questions failed six times in ten. So they wrote 470 graded test questions and built a loop on a cloud machine: deploy to QA through CI, ask every question as real test users, grade each answer, diagnose each failure to one cause, have one Claude Opus coding agent make the smallest fix with a unit test, and have a second Opus agent review the diff without seeing the first one's reasoning. Approved fixes were deployed and failures re-run, up to five rounds a night.
How it went
A 17-question pilot passed 2. After two days of looping the agent passed 435 of 470, or 435 of 441 leaving out cases broken inside QA itself. The loop made 8 commits to server code and 4 rounds of prompt changes, and none had to be reverted. Most fixes were clearer errors and inputs that accept names as well as numbers; nine reports that returned nothing started returning data. The reviewer caught a fix that would have let a gate report run with no site scope. The whole project took one week, and the team's job shrank to deciding what right looks like and reading the morning report.
Worth knowing
Rules were set before the first run: a read-only database user, the email tool disabled, no edits to test questions or graders, no new tools, no touching auth or deploy scripts, commits only to a feature branch and deploys only to QA. Expect a graded agent to try to fix a failing test by changing the test. The post names the models but not the coding agent harness.