Build and deploy a streaming serverless chat backend on AWS from a spec

Awan Shrestha gave Claude Code the same spec twice, with and without the Agent Toolkit for AWS. Both runs deployed a working streaming chat app in about 20 minutes; the toolkit run used 30k tokens to 45k.

Done withClaude CodeinAWS· Nov 21, 2026

What they did

Shrestha had built a serverless AI chat app on AWS by hand the year before. This time they wrote a SPEC.md for the same kind of system: a WebSocket API Gateway, Lambda, streamed responses from a model on Amazon Bedrock, DynamoDB for session history, everything in AWS CDK and least-privilege IAM. Claude Code built and deployed it twice from a fresh folder in the same account and region, first on its own, then with the aws-core plugin from the Agent Toolkit for AWS, which adds AWS skills and an MCP server. They only stepped in when it was stuck, logged every intervention, tested the results by hand over WebSocket and destroyed each stack before the next run.

How it went

Both runs deployed, streamed tokens and kept history across reconnects. The plain run took 21 minutes and the toolkit run 20. The toolkit run used about 30k tokens against 45k and needed only nudges, while the plain run needed one hard intervention. Both hit the same Bedrock error, caused by the spec naming the model directly instead of an inference profile, and both then had to widen IAM to three regions. The toolkit run found the fix in a skill rather than from error messages, and came out ahead on 5 of 12 implementation checks. Neither handled clients disconnecting mid-stream or set session expiry.

Worth knowing

Name the Bedrock inference profile ID, not the bare model ID, and expect cross-region IAM permissions to follow. The toolkit installs globally, so run any comparison without it first. The author calls it a small field test, not a benchmark.

Try it

With Claude Code· 4 parts to fill in

Read [SPEC.md] and build the system it describes on AWS in [region], with all infrastructure in [AWS CDK or my IaC tool] and least-privilege IAM. Deploy it with one command and give me one command to destroy it. When a deploy fails, read the error and fix it yourself; ask me only if you need information you cannot get from AWS. Keep a RESULT.md logging every error, fix and time I had to step in. Stop when [the acceptance tests in the spec] pass against the deployed stack, then list anything the spec missed that production would need.

Square brackets are the parts to fill in. The prompt is ours; the story is theirs.

Where they said it

Related