Artificial Intelligence
OpenAI reports benchmark gains for legacy code migration
OpenAI said its newer models handled older programming languages better and posted higher scores on a benchmark it built to test code generation and migration across 20 languages.
- OpenAI said its newer models were better at writing and migrating software in older languages such as COBOL, Fortran and Visual Basic.
- The company said it created MultiLangBench to measure progress by having models build a SQLite-compatible database engine from scratch across 20 languages.
- OpenAI said GPT-6 Astra averaged 87.4% on the benchmark, up from 26.8% for GPT-5.6 Sol.
- It said GPT-6.1 Sol reached 86.6% accuracy on COBOL migration at low reasoning effort and 97.9% at higher effort.
- OpenAI said the work could help companies modernise older software, but production migrations would still need expert review and testing.
OpenAI said its newer models improved on tasks involving COBOL, Fortran and Visual Basic, which it cited as examples of older programming languages. It said those languages had less training material available than more common languages.
The company said it built MultiLangBench to measure progress across 20 programming languages. The benchmark asked models to create a SQLite-compatible database engine from scratch.
OpenAI said GPT-6 Astra averaged 87.4%, compared with 26.8% for GPT-5.6 Sol. It said the gains appeared across every language it tested, including COBOL, OCaml, Fortran and Visual Basic.
OpenAI also said GPT-6.1 Sol performed better on migration tasks from COBOL into C++, Java, Kotlin, Python or Rust. It said the model reached 86.6% accuracy at low reasoning effort and 97.9% at higher effort, while noting that real migrations would still need specialist review and testing.
Named in this story
Companies
- OpenAI
- the company that said its newer models performed better on older languages
Products and systems
- GPT-6 Astra
- the model that OpenAI said averaged 87.4% on MultiLangBench
- GPT-5.6 Sol
- the earlier model used as the comparison baseline
- GPT-6.1 Sol
- the model that OpenAI said improved on COBOL migration tasks
- MultiLangBench
- the benchmark OpenAI built to test code generation and migration
- COBOL
- one of the older languages OpenAI cited and tested
- Fortran
- one of the older languages OpenAI cited and tested
- Visual Basic
- one of the languages OpenAI said showed improvement
- OCaml
- one of the languages OpenAI said improved on the benchmark
- C++
- one of the target languages in the migration test
- Java
- one of the target languages in the migration test
- Kotlin
- one of the target languages in the migration test
- Rust
- one of the target languages in the migration test
- SQLite
- the database system used as the benchmark target
How the source tells it
The piece was mostly straight reporting, but it leaned slightly towards vendor boosterism by foregrounding OpenAI's own performance claims and benchmark framing.
- vendor boosterism the article centred OpenAI's own benchmark results and product claims without independent verification