Humanity’s Sixth Sense (Scale AI and Elorian)
People score 93.1% on intuitive visual reasoning, the best model (GPT-6 Astra) 53.6%
Confirmed
The takeaway
On Oct 8, 2026 Scale AI and the startup Elorian published Humanity’s Sixth Sense (HSS), a benchmark of 522 open-ended questions about what an image or short video implies without showing it.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 2 of 5
- Last verified
- 13 October 2026
Your AI and this story
- GPT-6 Astra161 days after its cutoff
- Claude Opus 5.5100 days after its cutoff
- Gemini 4 Argon191 days after its cutoff
- Grok 4.7130 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 100 days before it.
Key facts
- Paper: ‘Humanity’s Sixth Sense: Benchmarking Intuitive Visual Reasoning in Multimodal Models’ (Guo, Gu, Jang et al.), a joint paper by Scale AI and Elorian; blog post dated Oct 8, 2026; public leaderboard and dataset
- Content: 522 open-ended tasks on 288 images and 234 short video clips (17.6 hours of video) in four areas: temporal and causal dynamics, physical and spatial logic, social understanding, abstract and contextual inference. Graded against human-written rubrics; an answer passes only if it meets every rubric point
- Human baseline: 93.1% pass@1 (94.1% on images, 91.9% on video)
- Top scores (pass@1, vendor-chosen effort settings): GPT-6 Astra 53.6, GPT-6.1 Sol 46.6, Claude Opus 5.5 44.6, Gemini 3.8 Flash 41.6, Claude Fable 5.1 40.8, Gemini 3.7 Flash 39.8, Muse Spark 1.3 37.4, Claude Fable 5 34.5, Qwen 3.8 Max 33.0. Lower: GPT-6 Sol 31.2, GLM-5.3 Flash 26.8, Kimi K3 25.5, GPT-6 Luna 21.0. Margins are about ±3 to ±4 points
- Not tested: Gemini 4 Argon and Meta’s Muse Wonder (both released to the public after the paper)
- Control: GPT-6 Astra with the image or video removed scores 6.6%, so the questions cannot be answered from text alone
- Failure analysis over 8,573 labeled failures: 53% misread the image, 41% failed to infer what was implied, 5% faulty logic. Most common single mistakes: missing a decisive detail (21%) and misidentifying an object, person or role (20%)
- Social understanding was the weakest area for 21 of 25 models (average 24.4% vs 34.1% in the other three); video was harder than images for 23 of 25 models, by 7.3 points on average
Show 1 more
- More reasoning did not always help: GPT-6 Astra lost 14 points on retrodiction tasks going from high to xhigh effort. With crop-and-zoom tools an agent reached 59.3% on a 388-task subset
What happened
Scale AI and Elorian, a startup that describes itself as building “the foundation of visual thinking”, released a benchmark of questions whose answers depend on what a scene implies: what happened just before, who is in charge, what will fall next. People answer almost all of them correctly at the first try. The best model, GPT-6 Astra at maximum reasoning effort, passes just over half, and most of the 25 models tested pass fewer than 40%.
The failure analysis says the gap is perceptual: in 94% of labeled failures the model misread the picture or did not infer what it implied, and in only 5% was the reasoning itself wrong. Social scenes were the hardest area and video harder than still images.
Only Elorian’s blog post was read for this entry (scores are those of Table 1 as reproduced there); the paper PDF, the leaderboard and the dataset page were not opened. We found no description of Elorian’s founders or funding on the page.
Why it matters
In the weeks when models were posting research-level mathematics and beating experts at ML experiment design, this benchmark shows a plain human skill where the best system is 40 points below an average person. The public dataset means the gap can be re-measured on Gemini 4 Argon and Muse Wonder.
Sources
5 sources from 5 sites. Numbers match the chips in the text.
5 sources: 4 primary, 1 press
Primary
- Elorian: Humanity’s Sixth Sense (Oct 8, 2026)elorian.ai, official
- Scale AI Labs: Humanity’s Sixth Sense (paper)labs.scale.com, paper
- Scale AI: HSS leaderboardscale.com, docs
- Hugging Face: ScaleAI/HSS datasethuggingface.co, code
Press
- Import AI 476 (Jack Clark, Oct 12, 2026)importai.substack.com, press
Changes
- Filed (found through Import AI 476)