AI legacy code migration: the proof is the hard part

A paper posted to arXiv on July 30 makes a point worth stealing for any legacy code migration: getting an AI model to write the new code is the easy half. Proving the new code behaves like the old code is the project. Agentic Method for Deterministic Validation of Legacy Code Migration, by Andras Ferenczi, Jordan Docherty, Mariya Bessonov, Matthew Findlay and Krishna Lingamneni, is not another COBOL-to-Java translator. It's the harness that tells you whether the translation was right.

What actually happened

The authors call it the Locksmith Loop. It instruments both the COBOL source and the generated Java target, synthesizes tests iteratively to push execution into branches nothing has covered yet, runs both versions, and compares outputs under deterministic parity checks. Where a branch can't be reached at all, the loop says so — which is useful on its own, because dead COBOL branches are code you can stop porting.

The reported numbers: near-complete branch coverage on two open-source programs, and 91.90% branch coverage on an internal production-like COBOL program. In every accepted test case, the generated Java matched the COBOL reference under the parity checks.

Notice what the paper does not claim. There's no assertion that the model's first-pass Java was correct. The contribution is the mechanism that catches it when it isn't, automatically, at a coverage level a human test-writing effort would take months to reach.

Why legacy code migration matters for your business

You almost certainly don't run COBOL. You do run something nobody wants to open: an Access database driving order intake, a PHP 5 admin panel, a spreadsheet with 40 macros that calculates commissions and has never been wrong. And in the last year somebody has told you AI can rewrite it in a weekend.

It might write it in a weekend. The question that decides whether the project ships is different: how do you know the new one does the same thing? Legacy systems don't have specifications. They have behavior. Every rounding quirk, every off-by-one, every weird Tuesday case that accounting silently adapted to eleven years ago — that is the spec, and it isn't written down anywhere.

So build the differential harness first. Run old and new side by side against real historical inputs, compare outputs field by field, and work the divergences down to zero. It's unglamorous and it's the only thing that ever gets a migration signed off by the people whose numbers have to keep matching. If somebody quotes you an AI-assisted rewrite and the quote has no parity test suite in it, they've priced half the job and you'll pay for the other half in production.

Sources: arXiv:2607.28271.