What they did
Shrestha had built a serverless AI chat app on AWS by hand the year before. This time they wrote a SPEC.md for the same kind of system: a WebSocket API Gateway, Lambda, streamed responses from a model on Amazon Bedrock, DynamoDB for session history, everything in AWS CDK and least-privilege IAM. Claude Code built and deployed it twice from a fresh folder in the same account and region, first on its own, then with the aws-core plugin from the Agent Toolkit for AWS, which adds AWS skills and an MCP server. They only stepped in when it was stuck, logged every intervention, tested the results by hand over WebSocket and destroyed each stack before the next run.
How it went
Both runs deployed, streamed tokens and kept history across reconnects. The plain run took 21 minutes and the toolkit run 20. The toolkit run used about 30k tokens against 45k and needed only nudges, while the plain run needed one hard intervention. Both hit the same Bedrock error, caused by the spec naming the model directly instead of an inference profile, and both then had to widen IAM to three regions. The toolkit run found the fix in a skill rather than from error messages, and came out ahead on 5 of 12 implementation checks. Neither handled clients disconnecting mid-stream or set session expiry.
Worth knowing
Name the Bedrock inference profile ID, not the bare model ID, and expect cross-region IAM permissions to follow. The toolkit installs globally, so run any comparison without it first. The author calls it a small field test, not a benchmark.