OpenAI COBOL migration test: 5.4% to 86.6% in one model

OpenAI published results on October 19 that move legacy code migration from "demo" to "line item." On its own COBOL migration test, GPT-6.1 Sol got 86.6% of programs right at low reasoning effort. The previous model, GPT-5.6 Sol, got 5.4%. The new model also cost 39% less to run. If you have an old Visual Basic app or an Access database that runs your business, this news is about you, not only about banks.

What actually happened

From OpenAI's post, Advancing coding in low-resource programming languages:

• COBOL migration: The model moves COBOL programs to C++, Java, Kotlin, Python, or Rust. Hidden tests require byte-for-byte parity on outputs. A migration passes only if every test passes. GPT-6.1 Sol: 86.6% at low effort, 97.9% at higher effort. GPT-5.6 Sol: 5.4%, at about 1.6x the API cost.
• MultiLangBench: The model builds a SQLite-compatible database engine from scratch in 20 languages. GPT-6 Astra averaged 87.4%, up from 26.8%. Visual Basic went from 27.5% to 89.4%. Fortran went from 21.4% to 83.5%. COBOL is still the hardest at 72.1%.
• Cheaper tier: GPT-6 Luna reached 40.3% on MultiLangBench at about $0.77 per sample, against $3.48 for GPT-5.6 Sol.
• Open test suite: OpenAI is releasing MultiLangBench and its tests on GitHub.
• OpenAI's caveat: Production migrations "still require expert review and testing against the system's business rules."

These are OpenAI's numbers on OpenAI's tests. Nobody has checked them independently yet.

Why legacy code migration matters for your business

The best line in the post is about rounding. COBOL drops extra decimal digits unless the code asks for rounding. A rewrite that rounds instead runs clean and gives you different invoice totals. There's no crash and no error message. The numbers are just wrong, and you find out from your accountant.

That is the real risk in every legacy migration, COBOL or not. The translation is now the cheap part. What costs money is proving the new code acts like the old code. We wrote about this in August, and this result makes the point stronger.

So do it in this order:

1. Capture real behavior first. Run last quarter's real inputs through the old system and save every output.
2. Make parity the test. OpenAI's bar is right: every output must match. "Close" is a bug.
3. Then let the model translate. At these prices, you can run it many times and keep the version that passes.

Sources: OpenAI.