{"schema":"postcutoff/event@1","as_of":"2026-10-14T07:26:00+02:00","url":"https://postcutoff.com/e/2026-10-09-theophyseval-gpt-6-astra-98-percent/","md":"https://postcutoff.com/e/2026-10-09-theophyseval-gpt-6-astra-98-percent/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-10-09-theophyseval-gpt-6-astra-98-percent","date":"2026-10-09","date_precision":"day","short_title":"TheoPhysEval: GPT-6 Astra in Codex solves 49 of 50 unpublished research-level theoretical-physics problems (98%), Opus 5.5 in Claude Code 92%","deck":null,"takeaway":"Thirty-seven physicists wrote 50 new problems that would take a world expert about a day each. Off-the-shelf coding agents solved nearly all of them for under $3 apiece, and barely needed web access.","category":"benchmark","category_label":"Benchmarks","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"arXiv 2610.16733: TheoPhysEval","url":"https://arxiv.org/abs/2610.16733","type":"paper","group":"primary","domain":"arxiv.org"}],"official":1,"filed":"2026-10-14","updated":"2026-10-14","orgs":["ICTS-TIFR","OpenAI","Anthropic","Google DeepMind"],"title":"TheoPhysEval: GPT-6 Astra in Codex solves 49 of 50 unpublished research-level theoretical-physics problems (98%), Opus 5.5 in Claude Code 92%","summary":"arXiv 2610.16733 (9 Oct 2026; Suvrat Raju, Praneeth Netrapalli and 35 co-authors from ICTS, TIFR, Harvard, Penn, UCSD, Wisconsin, KU Leuven, Google DeepMind and others) introduces TheoPhysEval: 50 unpublished research-level theoretical physics problems with verifiable answers. With web search and scripting (\"realistic track\"), GPT-6 Astra in Codex solved 49/50 (98%) at under $3 per problem; Claude Opus 5.5 in Claude Code 46/50; Gemini 3.8 Flash 34/50; DeepSeek-V4.1-Flash 27/50; GLM 5.3 21/50. Accuracy barely changed without internet, which the authors read as memorization of the literature.","key_facts":["Paper: 'TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving', A. Ramesh Chandra et al. (37 authors; coordinators Praneeth Netrapalli, Google DeepMind, and Suvrat Raju, ICTS), arXiv 2610.16733 [hep-th], submitted 9 Oct 2026","Problems: 50 unpublished problems with verifiable answers; 'a typical problem … could be solved by a world expert in about a day or less but would take considerably longer for experts in adjacent specialties'","Realistic track (web + scripts): GPT-6 Astra with Codex 49/50 (98%), Claude Opus 5.5 with Claude Code 46/50 (92%), Gemini 3.8 Flash with Antigravity 34/50, DeepSeek-V4.1-Flash with Codex 27/50, GLM 5.3 with Claude Code 21/50; max reasoning effort ('high' for Gemini)","Isolated track (Docker sandbox, no internet): Astra and Opus both 92%; 'The average change in accuracy for most agents between the two tracks was quite small', which 'indicates that leading LLMs have effectively memorized large parts of the theoretical physics literature'","Cost: Astra averaged under USD 3 per problem","Authors' reading: 'some frontier models may have already surpassed the best human researchers in their ability to perform concrete calculations'; they also discuss 'the current lack of agent-led breakthroughs in theoretical physics despite their outstanding problem-solving abilities'","AI use in the paper: agents including Astra, Gemini and Opus wrote evaluation code, analyzed results and assisted with the manuscript"],"key_numbers":[],"tags":["benchmark","physics","theoretical-physics","agents","gpt-6-astra","claude-opus-5-5","gemini-3-8-flash","deepseek","glm","codex","claude-code"],"science":null,"body_md":"## What happened\n\nOn 9 Oct 2026 a group of 37 physicists posted [arXiv 2610.16733](https://arxiv.org/abs/2610.16733) (listed 13 Oct). Expert contributors wrote 50 unpublished problems from many areas of theoretical physics, each with a checkable final answer. Five off-the-shelf agents were run without hints on two tracks: \"realistic\" (web search, literature, scripts) and \"isolated\" (no internet).\n\nGPT-6 Astra in Codex solved 49 of 50 on the realistic track at under $3 per problem, ahead of Claude Opus 5.5 in Claude Code (46), Gemini 3.8 Flash in Antigravity (34), DeepSeek-V4.1-Flash (27) and GLM 5.3 (21). Without internet, Astra and Opus both scored 92%. The authors conclude that frontier models have memorized much of the literature and may already beat the best humans at concrete calculations, while noting that agents have not yet produced breakthroughs in the field.\n\nNo reactions were found on 14 Oct (web search; X not searched directly).\n\n## Why it matters\n\nMost physics benchmarks reuse textbook or exam problems; these are new research-level tasks written by specialists. A near-saturated score at release, with small dependence on web access, says the bottleneck for AI in theoretical physics is no longer calculation. It also means the benchmark gives little room to separate future models.","disputed":[],"related":[{"id":"2026-10-13-ai-assisted-results-october-2026","url":"https://postcutoff.com/e/2026-10-13-ai-assisted-results-october-2026/","date":"2026-10-13","date_precision":"day","short_title":"October 2026: the arXiv stream of conjectures settled with disclosed AI help continues","deck":null,"takeaway":"In the first nine days of October 2026 the stream did not slow down: each arXiv listing brought several named conjectures with a disclosed AI role, and two new patterns appeared.","category":"science","category_label":"Science & math","importance":3,"confidence":"medium","status":{"key":"pending","labels":["Awaiting review"]},"sources":16,"official":16,"filed":"2026-10-13","updated":"2026-10-14","orgs":["OpenAI","Anthropic","Google DeepMind","various mathematicians"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-10-14","type":"filed","text":"Created from the arXiv paper"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-10-14","run":null,"sources_read":"The arXiv paper","updated":"2026-10-14","human_review":null,"version":null},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":162,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":101,"in_training_data":false},{"model_id":"gemini-4-argon","name":"Gemini 4 Argon","cutoff":"2026-03","days_after":192,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":131,"in_training_data":false}],"short_url":null}