Artificial Analysis launches Intelligence Index v5
60% private tests, FrontierCode 2.0 and Terminal-Bench-Science added; open-weights gap widens to 12 points
Confirmed
The takeaway
On Oct 27, 2026 Artificial Analysis replaced its widely cited Intelligence Index v4 with v5, its biggest change since January: nine evaluations in four equal groups (knowledge work, coding, science, general), 60% of them on private or held-out test sets.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 3 of 5
- Last verified
- 29 October 2026
Your AI and this story
- GPT-6.2 Astra149 days after its cutoff
- Claude Opus 5.5119 days after its cutoff
- Gemini 4 Hydrogen210 days after its cutoff
- Grok 4.7149 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 119 days before it.
Key facts
- New in v5: FrontierCode 2.0 (Cognition’s private benchmark of 132 tasks in real open-source repositories, graded on whether a maintainer would merge the change; launched the same day) and Terminal-Bench-Science (70 agentic research tasks in a terminal). Upgraded: HLE-Diamond (1,000-question refined Humanity’s Last Exam), AA-Omniscience v1.1, CritPt v2 private 40-question hard subset. Kept: AA-Briefcase, AutomationBench-AA, GDP.pdf, Terminal-Bench 4.0. Removed: GDPval-AA v2.1, SciCode, AA-LCR v1.1
- Weights: knowledge work 25% (AA-Briefcase 15, AutomationBench-AA 5, GDP.pdf 5); coding 25% (FrontierCode 2.0 15, Terminal-Bench 4.0 10); science 25% (Terminal-Bench-Science 15, CritPt v2 hard 10); general 25% (AA-Omniscience v1.1 15, HLE-Diamond 10)
- Launch results (Oct 27): Claude Opus 5.5 (max) 58, GPT-6 Astra (max) 57, GPT-6.1 Sol (max) 55; Gemini 4 Argon (high) 50; Anthropic and OpenAI held the top five. Kimi K3.1 Pro 46 is the top open-weights model (weights not yet released); among released weights DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32, below Claude Haiku 5.5 (max, 36). The open-weights gap grew from 3 points (v4.3) to 12
- Per-benchmark leaders: Opus 5.5 on FrontierCode 2.0 (46%), HLE-Diamond (68%) and AA-Omniscience v1.1 (48); GPT-6 Astra on Terminal-Bench-Science (63%) and CritPt v2 hard (72%); Sonnet 5.5 on AA-Briefcase (1812 Elo); Kimi K3.1 Pro on Terminal-Bench 4.0 (67%); Argon on AutomationBench-AA (78%); Muse Wonder Full on GDP.pdf (33% all-pass)
- Cost per index task: GPT-6.1 Terra (max) $0.96 to Claude Fable 5.1 (max) $13.04 among the top ten; Opus 5.5 (max) $10.04, GPT-6 Astra (max) $6.24, GPT-6.1 Sol (max) $1.27. Opus 5.5 scores 35 at low effort and 58 at max (10x the cost); median output tokens per task rose from 35k to 59k
- Oct 28: Gemini 4 Carbon (high) entered at 59, #1 of 52 models (see Google announces Gemini 4 Carbon, #1 on the Artificial Analysis Intelligence Index v5 (59), two weeks after Argon; first only for Fairwind cyber defenders)
- Scores are not comparable with v4.x numbers quoted in older entries (Argon 53 on v4, 50 on v5; Kimi K3.1 Pro 55 to 46; GPT-6 Astra 53 to 57)
What happened
Artificial Analysis, whose composite index labs quote in launch posts (Google led its Carbon announcement with it), moved to a harder v5 built mostly on private tests to limit contamination. Several agentic tasks now dominate token use: Terminal-Bench-Science alone accounts for about a third of the output tokens per index task.
Why it matters
Leaderboard positions in late October 2026 shifted because the yardstick changed, not only because models did. The new index puts more weight on long agentic work and private coding tasks, where open-weights models lag most, so the measured open-closed gap quadrupled overnight.
Sources
4 sources from 1 site. Numbers match the chips in the text.
4 sources: 4 primary
Primary
- Artificial Analysis: Announcing Artificial Analysis Intelligence Index v5 (Oct 27)artificialanalysis.ai, official
- Artificial Analysis: Intelligence Index methodologyartificialanalysis.ai, docs
- Artificial Analysis Progress Index (trends)artificialanalysis.ai, docs
- Artificial Analysis Intelligence Index leaderboardartificialanalysis.ai, docs
Changes
- Filed (seen on Techmeme)