Artificial Intelligence
OpenAI says GPT-6 improves on COBOL and Fortran
OpenAI said its latest models performed better on tests for COBOL, Fortran and Visual Basic, with GPT-6 variants outperforming GPT-5.6 models on two benchmarks the company built.
- OpenAI said it had improved its models for COBOL, Fortran and Visual Basic.
- The company said IBM estimated COBOL powered more than 40 percent of online banking systems and 95 percent of ATM transactions.
- OpenAI built MultiLangBench to test whether models could create a SQLite-compatible database engine.
- GPT-6 Astra scored 87.4 percent across 20 languages on MultiLangBench at maximum reasoning effort, versus about 38 percent for GPT-5.6 Terra.
- GPT-6.1 Sol scored 97.9 percent on the COBOL migration benchmark at higher reasoning effort, compared with 5.4 percent for GPT-5.6 Sol at low effort.
OpenAI said it had improved its models for low-resource languages including COBOL, Fortran and Visual Basic. It also repeated IBM’s estimate that COBOL powered more than 40 percent of online banking systems and 95 percent of ATM transactions.
The company built MultiLangBench to measure progress on code generation across languages. The benchmark asked models to build a SQLite-compatible database engine from scratch, and GPT-6 Astra reached an average pass rate of 87.4 percent across 20 languages at maximum reasoning effort.
OpenAI said GPT-5.6 Terra had reached about 38 percent on the same test. GPT-6 Luna scored 40.3 percent, slightly above GPT-5.6 Terra’s 37.9 percent, while costing less.
On a second benchmark, GPT-6.1 Sol translated COBOL into C++, Java, Kotlin, Python or Rust. It scored 86.6 percent at low reasoning effort and 97.9 percent with higher reasoning, against 5.4 percent for GPT-5.6 Sol at low effort.
Named in this story
Companies
- OpenAI
- said it had improved its latest models for low-resource languages
- IBM
- was cited for estimating COBOL’s use in banking and ATM systems
Products and systems
- COBOL
- was one of the low-resource languages OpenAI discussed
- Fortran
- was one of the low-resource languages OpenAI discussed
- Visual Basic
- was one of the low-resource languages OpenAI discussed
- MultiLangBench
- was the benchmark OpenAI built for cross-language coding tasks
- GPT-6 Astra
- scored 87.4 percent on MultiLangBench
- GPT-5.6 Terra
- was the earlier model compared with GPT-6 Astra and GPT-6 Luna
- GPT-6 Luna
- scored 40.3 percent on MultiLangBench at lower cost
- GPT-6.1 Sol
- scored 86.6 percent and 97.9 percent on the COBOL migration benchmark
- GPT-5.6 Sol
- was the earlier model compared with GPT-6.1 Sol
How the source tells it
The piece was mostly a neutral benchmark report, with a mild vendor-boosterish cast from OpenAI’s own comparison framing.
- vendor boosterism the headline and benchmark results repeated OpenAI’s self-assessment of improvement