{"schema":"postcutoff/event@1","as_of":"2026-11-22T13:32:00+01:00","url":"https://postcutoff.com/e/2026-09-22-hle-diamond/","md":"https://postcutoff.com/e/2026-09-22-hle-diamond/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-09-22-hle-diamond","date":"2026-09-22","date_precision":"day","short_title":"CAIS releases HLE-Diamond, a refined 1,000-question Humanity's Last Exam","deck":"GPT-6 Astra scores 82.9% with tools","takeaway":"Around Sept 22, 2026 the Center for AI Safety released HLE-Diamond, a cleaned 1,000-question subset of Humanity's Last Exam (500 reasoning, 500 knowledge).","category":"benchmark","category_label":"Benchmarks","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"CAIS / lastexam.ai: HLE-Diamond","url":"https://lastexam.ai/blog/hle-diamond","type":"official","group":"primary","domain":"lastexam.ai"},{"n":2,"title":"Scale Labs leaderboard: HLE-Diamond","url":"https://labs.scale.com/leaderboard/hle-diamond","type":"docs","group":"primary","domain":"labs.scale.com"},{"n":3,"title":"Octagon AI: Humanity's Last Exam market odds","url":"https://www.octagonai.co/news/ai-benchmark-humanity-s-last-exam-market-odds/","type":"press","group":"press","domain":"octagonai.co"}],"official":2,"filed":"2026-09-29","updated":"2026-09-30","orgs":["Center for AI Safety","Scale AI"],"title":"CAIS releases HLE-Diamond, a refined 1,000-question Humanity's Last Exam; GPT-6 Astra scores 82.9% with tools","summary":"Around Sept 22, 2026 the Center for AI Safety released HLE-Diamond, a cleaned 1,000-question subset of Humanity's Last Exam (500 reasoning, 500 knowledge). GPT-6 Astra scored 82.9% with web and code tools (59.9% without), the first HLE-series result above 75–80%. The prediction-market contract on a ≥75% score by end-2026 jumped from 14% to 87%.","key_facts":["HLE-Diamond: 1,000 questions (500 reasoning + 500 knowledge), a refined subset of HLE","Confirmed on lastexam.ai (checked 2026-09-30): GPT-6 Astra 59.9% without tools and 82.9% with tools (web + code)","lastexam.ai no-tools chart (reasoning high): GPT-6 Astra 59.9%, Claude Opus 5.5 54.6%, GPT-6.1 Sol 53.2%, Claude Fable 5.1 50.7%, Gemini 3.8 Flash 33.3%, Claude Sonnet 5.5 33.1%, GPT-6 Sol 32.8%, Muse Spark 1.3 24.6%, Grok 4.7 22.8%","Scale Labs leaderboard (no tools): gpt-6-astra 60.6% ±3.0, claude-opus-5-5 55.0%, claude-fable-5-1 51.3%, claude-opus-5 38.6%, gemini-3.8-flash 34.3%, gpt-6-sol 33.8%, kimi-k3 22.2%, deepseek-v4-pro 13.4%","Previous best on the main HLE reported at about 65% (Claude Fable 5.1)","Kalshi 'HLE ≥75% by end of 2026' market moved from 14% to 87%","HLE-Rolling was updated on Sept 17, 2026"],"key_numbers":[],"tags":["benchmark","humanitys-last-exam","reasoning","evaluation"],"science":null,"body_md":"## What happened\n\nCAIS produced a higher-quality core of HLE after criticism that some original questions had wrong or ambiguous answers. The first reported frontier score is far above the main-HLE scores from earlier in 2026.\n\n## Why it matters\n\nIf confirmed, the benchmark once billed as 'the last exam' is close to saturation within about 20 months of release. The 82.9% with-tools score is confirmed on the CAIS page; without tools the best model is near 60%.","disputed":[],"related":[{"id":"2026-09-03-gpt-6-astra","url":"https://postcutoff.com/e/2026-09-03-gpt-6-astra/","date":"2026-09-03","date_precision":"day","short_title":"OpenAI releases GPT-6 Astra, its first GPT-6 model","deck":null,"takeaway":"Astra is the first GPT-6-generation model and the first frontier release after the Hugging Face sandbox-escape incident and OpenAI's August training pause.","category":"model-release","category_label":"Model releases","importance":5,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":20,"official":11,"filed":"2026-09-29","updated":"2026-10-19","orgs":["OpenAI"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-09-30","type":"edit","text":"Confirmed GPT-6 Astra 82.9% (with tools) and 59.9% (no tools) on lastexam.ai; added full no-tools chart and Scale leaderboard; confidence raised to high"},{"date":"2026-09-29","type":"filed","text":"Created"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-29","run":null,"sources_read":null,"updated":"2026-09-30","human_review":null,"version":{"date":"2026-10-04"}},"gaps":[{"model_id":"gpt-6-2-astra","name":"GPT-6.2 Astra","cutoff":"2026-05","days_after":114,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":84,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":175,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":114,"in_training_data":false}],"short_url":null}