{"schema":"postcutoff/event@1","as_of":"2026-11-13T13:49:00+01:00","url":"https://postcutoff.com/e/2024-12-20-frontiermath-o3-25-percent/","md":"https://postcutoff.com/e/2024-12-20-frontiermath-o3-25-percent/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2024-12-20-frontiermath-o3-25-percent","date":"2024-12-20","date_precision":"day","short_title":"OpenAI o3 scores 25% on FrontierMath research-level maths benchmark, amid funding disclosure controversy","deck":null,"takeaway":"Epoch AI's FrontierMath (Nov 2024) contains unpublished research-level problems on which models scored under 2%.","category":"science","category_label":"Science & math","importance":3,"confidence":"high","status":{"key":"disputed","labels":["Disputed"]},"sources":[{"n":1,"title":"Epoch AI: OpenAI and FrontierMath","url":"https://epoch.ai/latest/openai-and-frontiermath","type":"official","group":"primary","domain":"epoch.ai"},{"n":2,"title":"The Decoder: OpenAI quietly funded independent math benchmark","url":"https://the-decoder.com/openai-quietly-funded-independent-math-benchmark-before-setting-record-with-o3/","type":"press","group":"press","domain":"the-decoder.com"},{"n":3,"title":"TechRepublic: independent FrontierMath score for o3","url":"https://www.techrepublic.com/article/news-openai-generative-ai-models-frontiermath-score/","type":"press","group":"press","domain":"techrepublic.com"}],"official":1,"filed":"2026-09-29","updated":"2026-09-29","orgs":["OpenAI","Epoch AI"],"title":"OpenAI o3 scores 25% on FrontierMath research-level maths benchmark, amid funding disclosure controversy","summary":"Epoch AI's FrontierMath (Nov 2024) contains unpublished research-level problems on which models scored under 2%. On 20 Dec 2024 OpenAI claimed 25.2% for o3. It then emerged that OpenAI had funded the benchmark and had access to most problems. Released o3 scored lower in independent tests.","key_facts":["FrontierMath paper v1: 7 Nov 2024; prior models <2%","o3 claimed 25.2% (aggressive test-time compute setting) on 20 Dec 2024","OpenAI's funding and data access disclosed only in paper v5 (20 Dec 2024); Epoch said it should have been more transparent","Later records: Gemini 3 Pro 38% (Tiers 1-3) and 19% (Tier 4) in Nov 2025; GPT-5.2 Pro 31% on Tier 4 in Jan 2026"],"key_numbers":[],"tags":["math","benchmark","frontiermath","controversy"],"science":{"field":"mathematics","subfield":"benchmarks","problem":"FrontierMath: unpublished research-level problems with automatically checkable answers","result":"Score jump from <2% to a claimed 25.2% in about six weeks, later partly qualified by independent evaluation.","open_since":"","ai_system":["OpenAI o3"],"human_role":"Autonomous answering; benchmark written by expert mathematicians","verification":{"key":"expert","label":"Expert-checked","text":"Company-reported; independent Epoch evaluation of released o3 was lower"},"status":"disputed","shock":"When FrontierMath launched, Fields medallists including Terence Tao said its problems would likely resist AI for years; a big jump came within weeks."},"body_md":"## What happened\n\nOpenAI previewed o3 with a headline FrontierMath score an order of magnitude above prior models. The benchmark's independence was then questioned when OpenAI's funding and access came to light.\n\n## Why it matters\n\nIt was the first sign that research-level maths was yielding to reasoning models, and an early lesson in benchmark governance and conflicts of interest.","disputed":[],"related":[{"id":"2026-05-09-deepmind-ai-co-mathematician","url":"https://postcutoff.com/e/2026-05-09-deepmind-ai-co-mathematician/","date":"2026-05-09","date_precision":"day","short_title":"Google DeepMind's 'AI co-mathematician' sets FrontierMath Tier 4 record and helps resolve a Kourovka Notebook problem","deck":null,"takeaway":"FrontierMath Tier 4, built to resist AI for years, was nearly half solved 18 months after launch.","category":"science","category_label":"Science & math","importance":3,"confidence":"high","status":{"key":"pending","labels":["Event confirmed","Awaiting review"]},"sources":3,"official":2,"filed":"2026-09-29","updated":"2026-10-19","orgs":["Google DeepMind","University of Oxford"]},{"id":"2024-12-20-openai-o3","url":"https://postcutoff.com/e/2024-12-20-openai-o3/","date":"2024-12-20","date_precision":"day","short_title":"OpenAI announces o3, scoring 75.7–87.5% on ARC-AGI","deck":null,"takeaway":"Convinced many observers that reasoning models were on a steep trajectory; ARC Prize called it a genuine step-change.","category":"benchmark","category_label":"Benchmarks","importance":5,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":2,"official":2,"filed":"2026-09-29","updated":"2026-09-29","orgs":["OpenAI","ARC Prize"]}],"people":[{"id":"terence-tao","name":"Terence Tao","url":"https://postcutoff.com/person/terence-tao/"}],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-09-29","type":"filed","text":"Created"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-29","run":null,"sources_read":null,"updated":"2026-09-29","human_review":null,"version":{"date":"2026-10-04"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":null,"in_training_data":true},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":null,"in_training_data":true},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":null,"in_training_data":true},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":null,"in_training_data":true}],"short_url":null}