Artificial Intelligence
Import AI roundup covers new benchmarks and model launches
Import AI collected several AI research and model announcements, including a visual-reasoning benchmark, a research-taste benchmark, new open-weight model releases, and Jack Clark’s revised view on when full RSI might become possible.
- Elorian and Scale AI released Humanity’s Sixth Sense, a benchmark for intuitive visual reasoning, with 522 questions across 288 images and 234 short video clips.
- The article said the human aggregate pass@1 on Humanity’s Sixth Sense was 93.1, while GPT 6 Astra scored 53.6.
- P-Zero Research introduced TasteVal, which it said measured research taste across frontier AI tasks, and the authors said Opus 5.5 exceeded their human baseline on the experimental measure.
- Reflection announced Beam, a 501 billion parameter mixture-of-experts model, and said it would later release the weights, technical report, model card and developer materials.
- Jack Clark said he now thought full recursive self-improvement was more likely in 2027 than in 2028.
Elorian and Scale AI released Humanity’s Sixth Sense, which asked models to infer what a scene implied rather than what it explicitly showed. The paper said the benchmark covered 522 questions across 288 images and 234 short video clips, and it reported a human aggregate pass@1 of 93.1 against 53.6 for GPT 6 Astra.
The newsletter also summarised an AI Frontiers article that argued AI could accelerate the industrialisation of space and create compounding advantages for first movers there. It said spacecraft could be designed for autonomous operation, that assembly and manufacturing in space could become machine-led, and that access to space could itself become a source of leverage.
P-Zero Research introduced TasteVal, a benchmark that tested how models designed experiments and drew conclusions in frontier research settings. Its authors said the best model, Opus 5.5, beat their human baseline on the experimental task, and Jack Clark said he now thought full recursive self-improvement was more likely in 2027 than in 2028.
Reflection then announced Beam, a 501 billion parameter mixture-of-experts model with 23 billion active parameters. Mistral also announced Mistral Large 4, said it would release it by the end of the month, and described it as an open-weight model tuned for cyber work.
Named in this story
People
- Jack Clark
- said full RSI was more likely in 2027 than 2028
Companies
- Elorian
- helped build and release Humanity’s Sixth Sense
- Scale AI
- helped build and release Humanity’s Sixth Sense
- Reflection
- revealed Beam and said it would release the model materials later in October
- Mistral
- announced Mistral Large 4 and planned an open-weight release
Organisations
- P-Zero Research
- introduced TasteVal
Products and systems
- Humanity’s Sixth Sense
- benchmark for intuitive visual reasoning
- TasteVal
- benchmark for research taste in frontier AI development
- Opus 5.5
- best model on TasteVal’s experimental research-taste measure
- Beam
- 501 billion parameter mixture-of-experts model
- Mistral Large 4
- open-weight model tuned for cyber tasks
- GPT 6 Astra
- best-scoring model named on Humanity’s Sixth Sense
- GLM-5.3-flash
- one of the low-scoring models reported on Humanity’s Sixth Sense
- Kimi K3
- one of the low-scoring models reported on Humanity’s Sixth Sense
How the source tells it
The newsletter mixed neutral benchmark reporting with overtly enthusiastic commentary and speculative future-gazing in its editorial asides.
- boosterism the author repeatedly frames linked work as important in editorial asides
- speculation presented as fact future claims about 2027, the 2030s and space dominance are stated with little qualification
- urgency time-pressure language about sooner-than-expected shifts and make-or-break decisions recurs