Migrate legacy DataStage ETL jobs to Dataform code on BigQuery

The data platform team at Unieuro, an Italian consumer electronics retailer, worked with Thoughtworks and Huware to modernize a legacy data estate that was only partly governed. They used the Gemini model through Google's agent platform (formerly Vertex AI), tested alongside the Antigravity coding agent against a test set before committing. In one stream the agent read DataStage ETL jobs, stored as hard to read proprietary XML with no version control or lineage, and rewrote them as Dataform code running on BigQuery, writing a business explanation and a technical document next to each job. In a second stream it moved metric logic out of Power BI DAX and QlikView scripts into a single Looker semantic layer kept in Git. The first pilot was one stock and availability ETL flow, run against production sources and shipped through the normal deployment process. Comparing outputs, running checks and deploying stay in a deterministic workflow, and a person signs off every change.

On the ETL stream the team reports that answering an audit question went from days of work to under an hour, and that a migrated job went from a quarter-sized project to a few weeks, with each agent translation reviewed and signed off within a working week. CI now regenerates the explanation and document whenever a job changes. For the BI stream they list the metrics they track but give no figures. The pilot left templates and written skills that other domain teams now reuse through an internal registry.

The team says the agent could not infer company specific definitions, so human review of both the code and its explanation was essential, and per pipeline validation suites (row counts, duplicates, column statistics) took real time to design up front. They also had a separate subagent check each explanation against its code. The post is written by the company with its consulting partners and is partly a methodology piece.