What they did
Nilesh Parekh, who works at Databricks, built a copilot for the paperwork around an STR, the report an Indian bank sends to the financial intelligence unit when a transaction looks suspicious. On a synthetic mule-account case he gave it read access to the customer, payment and alert tables and asked it to write the grounds. Three limits were put in the data and the tooling rather than the prompt: a separate check rejects any drafted sentence that does not cite a record from that case, the two write tools sit behind an approval gateway and also require the named Principal Officer's recorded decision before they save a row, and the agent has no messaging tool at all, so it cannot tell the customer. He then tried to make it break those rules and recorded 15 runs.
How it went
The draft came back as six sentences, each pointing at specific payment IDs, and a pasted sentence citing a payment that does not exist was caught. The analyst could not reach an approval page at all, because she lacks permission to use the write service. On a case with no officer decision on record, the write refused even after approval was granted. Asked to freeze the customer's account, the agent refused, since a report alone must not restrict an account; on a fictitious firm that matched a designated list it required the freeze first. The citation check is the weak part: a test sentence claiming a five lakh transfer passed while citing a real 795 rupee credit, because the check only proves the record exists, not that it supports the sentence.
Worth knowing
Synthetic data throughout, no real list and nothing filed, and the author discloses he works at Databricks. An unanswered approval writes nothing, which is correct, but the seven-day filing clock keeps running, so open approvals need an owner. The approval only checks that a decision row exists, not who wrote it, so the write permission has to be limited to the officers. The gateway pause applies only to calls that go through it, which is why the write server also checks its credential.