Memento 3 (UCL, Huawei Noah’s Ark Lab) clears all 25 public ARC-AGI-3 games with a frozen Claude Opus 5 that writes its own world model
Partly confirmed
Status
- Claim
Partly confirmed
- Our reporting
- Medium confidence
- Importance
- 3 of 5
- Last verified
- 12 October 2026
Your AI and this story
- GPT-6 Astra161 days after its cutoff
- Claude Opus 5.5100 days after its cutoff
- Gemini 3.8 Flash191 days after its cutoff
- Grok 4.7130 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 100 days before it.
Key facts
- Paper: ‘Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks’, arXiv 2610.11794v1, submitted Oct 8, 2026; authors Haoyu Zhao, Zhengxu Yu, Zhiyuan He, Meng Fang, Rasul Tutunov, Haitham Bou-Ammar, Weilin Luo, Jun Wang
- Method (‘Code as Model’): the agent keeps a rulebook of revisable hypotheses about the environment, compiles it into code for prediction and planning, and revises both from prediction errors; new code is accepted only when the LLM judges it faithful to the rulebook and ‘cell-exact replay reproduces the observed transitions’
- Setup: harness built from the ‘baseline1’ codebase, run ‘in Claude Code with Claude Opus 5 as the backbone model and reasoning effort set to extra-high’; the agent sees only the action interface, not the environment source
- Result: all 25 public games cleared, mean RHAE (Relative Human Action Efficiency) 100.0, 7,518 actions against a human baseline of 17,135 (0.44); per-game ratios from 0.20 to 0.67
- Comparison in the paper: 59.3 points higher mean RHAE than Claude Opus 5 with the ARC Prize standard harness (which implies 40.7)
- The gain over the best earlier harness is small: ‘baseline1’ also finishes every game, with 8,347 actions (0.49 of the human count) and a mean RHAE of 99.0 in the paper’s table; the other four community-leaderboard systems compared (OPINE-World, NOOA, DreamTeam, Continual Harness) leave 5, 6, 19 and 22 games unfinished (mean RHAE 78.4, 85.1, 38.1 and 20.5)
- Ablation on three games: the rulebook cuts actions by 9% (617 vs 677) and agent turns by 18% (678 vs 830); a two-model population on game wa30 cuts actions from 899 to 597
- Atari Pong case study: a learned feedback controller wins 21:0 in three evaluated episodes without LLM calls during play
Show 1 more
- Caveat stated by the authors: ‘Public-set completion alone therefore cannot isolate the contribution of a world-model architecture’; comparisons must also account for model choice, reasoning budget and held-out evaluation. No semi-private or ARC Prize-verified score is reported, and the paper gives no cost figure beyond ‘substantial time and monetary cost’
What happened
Memento 3 continues the Memento series of agents that learn through external memory instead of weight updates. Earlier versions stored experiences and reusable skills; this one learns the “model side”, the rules of an unfamiliar environment. The agent explores a game, writes what it believes about the dynamics into a Markdown rulebook, compiles the rulebook into a Python world model, plans with that model, and repairs both when a prediction fails. The authors present this as a model-based route to recursive self-improvement in which “the underlying LLM remains fixed”.
The numbers come from the paper (read from the PDF). Every figure for other systems is taken by the authors from those systems’ entries on the ARC-AGI-3 community leaderboard.
Why it matters
ARC-AGI-3’s public games are now solved at the efficiency ceiling by two harnesses built on the same frozen model, while that model alone scores about 41 with the standard harness. The difference comes from the scaffolding, which is the same pattern as in other recent harness results. The result does not show performance on held-out games, which is what the benchmark’s official scores measure.
Sources
3 sources from 3 sites. Numbers match the chips in the text.
3 sources: 1 primary, 2 press
Primary
- arXiv 2610.11794: Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooksarxiv.org, paper
Press
Changes
- Filed (Pandaily Oct 12)