TheoPhysEval: GPT-6 Astra in Codex solves 49 of 50 unpublished research-level theoretical-physics problems (98%), Opus 5.5 in Claude Code 92%
Confirmed
The takeaway
Thirty-seven physicists wrote 50 new problems that would take a world expert about a day each. Off-the-shelf coding agents solved nearly all of them for under $3 apiece, and barely needed web access.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 3 of 5
- Last verified
- 14 October 2026
Your AI and this story
- GPT-6 Astra162 days after its cutoff
- Claude Opus 5.5101 days after its cutoff
- Gemini 4 Argon192 days after its cutoff
- Grok 4.7131 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 101 days before it.
Key facts
- Paper: ‘TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving’, A. Ramesh Chandra et al. (37 authors; coordinators Praneeth Netrapalli, Google DeepMind, and Suvrat Raju, ICTS), arXiv 2610.16733 [hep-th], submitted 9 Oct 2026
- Problems: 50 unpublished problems with verifiable answers; ‘a typical problem … could be solved by a world expert in about a day or less but would take considerably longer for experts in adjacent specialties’
- Realistic track (web + scripts): GPT-6 Astra with Codex 49/50 (98%), Claude Opus 5.5 with Claude Code 46/50 (92%), Gemini 3.8 Flash with Antigravity 34/50, DeepSeek-V4.1-Flash with Codex 27/50, GLM 5.3 with Claude Code 21/50; max reasoning effort (‘high’ for Gemini)
- Isolated track (Docker sandbox, no internet): Astra and Opus both 92%; ‘The average change in accuracy for most agents between the two tracks was quite small’, which ‘indicates that leading LLMs have effectively memorized large parts of the theoretical physics literature’
- Cost: Astra averaged under USD 3 per problem
- Authors’ reading: ‘some frontier models may have already surpassed the best human researchers in their ability to perform concrete calculations’; they also discuss ‘the current lack of agent-led breakthroughs in theoretical physics despite their outstanding problem-solving abilities’
- AI use in the paper: agents including Astra, Gemini and Opus wrote evaluation code, analyzed results and assisted with the manuscript
What happened
On 9 Oct 2026 a group of 37 physicists posted arXiv 2610.167331 (listed 13 Oct). Expert contributors wrote 50 unpublished problems from many areas of theoretical physics, each with a checkable final answer. Five off-the-shelf agents were run without hints on two tracks: “realistic” (web search, literature, scripts) and “isolated” (no internet).
GPT-6 Astra in Codex solved 49 of 50 on the realistic track at under $3 per problem, ahead of Claude Opus 5.5 in Claude Code (46), Gemini 3.8 Flash in Antigravity (34), DeepSeek-V4.1-Flash (27) and GLM 5.3 (21). Without internet, Astra and Opus both scored 92%. The authors conclude that frontier models have memorized much of the literature and may already beat the best humans at concrete calculations, while noting that agents have not yet produced breakthroughs in the field.
No reactions were found on 14 Oct (web search; X not searched directly).
Why it matters
Most physics benchmarks reuse textbook or exam problems; these are new research-level tasks written by specialists. A near-saturated score at release, with small dependence on web access, says the bottleneck for AI in theoretical physics is no longer calculation. It also means the benchmark gives little room to separate future models.
Sources
1 source from 1 site. Numbers match the chips in the text.
1 source: 1 primary
Primary
- arXiv 2610.16733: TheoPhysEvalarxiv.org, paper
Changes
- Filed from the arXiv paper