On Oct. 19, OpenAI published results that put its GPT-6 models far ahead of GPT-5.6 Sol at porting COBOL. On a test that ports COBOL to modern languages, GPT-6.1 Sol scores 86.6% at low reasoning effort, against 5.4% for GPT-5.6 Sol.
What OpenAI tested
OpenAI built two evaluations for older and less common programming languages. These are OpenAI's own tests, and every figure here is OpenAI's.
- MultiLangBench: the model must build a SQLite-compatible engine in 20 languages.
- COBOL migration: the model ports COBOL code to C++, Java, Kotlin, Python or Rust. The new program must match the original's output byte for byte.
OpenAI points to why COBOL matters. It cites figures that the language underpins more than 40% of online banking systems, 80% of in-person card transactions and 95% of ATM transactions.
The results
On MultiLangBench, GPT-6 Astra averages an 87.4% pass rate. GPT-5.6 Sol averaged 26.8%. GPT-6 Astra is above 83% in 19 of the 20 languages. COBOL is the hardest.
| Language | GPT-5.6 Sol | GPT-6 Astra |
|---|---|---|
| COBOL | 10.9% | 72.1% |
| Fortran | 21.4% | 83.5% |
| Visual Basic | 27.5% | 89.4% |
| OCaml | 25.6% | 85.8% |
On the COBOL migration test, GPT-6.1 Sol reaches 86.6% at low reasoning effort. At higher effort it reaches 97.9%. GPT-5.6 Sol scored 5.4%.
Cost
OpenAI also compared price. GPT-6 Luna averages 40.3% on MultiLangBench at an estimated $0.77 per sample. GPT-5.6 Sol costs an estimated $3.48 per sample, so Luna is about 78% cheaper.
On COBOL migration, GPT-6.1 Sol at low effort costs 61% of what GPT-5.6 Sol does through the API, while scoring far higher.
What it means for teams with legacy code
Porting old COBOL systems to modern languages has been slow work. A model that can port code and match the old program's output could make it more practical, if the results hold up outside OpenAI's own tests.
The byte-for-byte rule is the useful part. It gives a team a clear check: run the old and new programs on the same input and compare the results.
For more on OpenAI's models, see our coverage of the GPT-6 Pro update and OpenAI's Simons Foundation funding.
Caveats
OpenAI is releasing the MultiLangBench setup and test suite on GitHub, so others will be able to run it. The numbers in this article are OpenAI's.
Parity on a test is not the same as a safe production migration. OpenAI itself says production use still needs expert review. An 86.6% score also means some migrations still fail, and a real banking system is larger and messier than a benchmark task.
