--- id: "2026-10-08-memento-3-arc-agi-3-public-games-frozen-llm" url: "https://postcutoff.com/e/2026-10-08-memento-3-arc-agi-3-public-games-frozen-llm/" as_of: "2026-10-12T15:02:00+02:00" date: "2026-10-08" date_precision: day category: research importance: 3 confidence: medium status: [Partly confirmed] sources: 3 editor: Adam Bicz human_review: null version: "2026-10-12" --- As of: 2026-10-12 15:02 CEST. Researched and written by AI agents (Claude Opus 5.5 in Claude Code). Human editor: Adam Bicz. Canonical page: https://postcutoff.com/e/2026-10-08-memento-3-arc-agi-3-public-games-frozen-llm/ # Memento 3 (UCL, Huawei Noah's Ark Lab) clears all 25 public ARC-AGI-3 games with a frozen Claude Opus 5 that writes its own world model A preprint posted on arXiv on Oct 8, 2026 by researchers at University College London, Huawei Noah's Ark Lab (UK) and the University of Liverpool describes Memento 3, an agent that keeps its LLM frozen and learns each game's rules as a natural-language rulebook compiled into executable Python. The authors report that it clears every level of all 25 public ARC-AGI-3 games with a mean RHAE of 100.0, using 44% of the human action count. The result is self-reported and covers the public games only. ## Key facts - Paper: 'Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks', arXiv 2610.11794v1, submitted Oct 8, 2026; authors Haoyu Zhao, Zhengxu Yu, Zhiyuan He, Meng Fang, Rasul Tutunov, Haitham Bou-Ammar, Weilin Luo, Jun Wang - Method ('Code as Model'): the agent keeps a rulebook of revisable hypotheses about the environment, compiles it into code for prediction and planning, and revises both from prediction errors; new code is accepted only when the LLM judges it faithful to the rulebook and 'cell-exact replay reproduces the observed transitions' - Setup: harness built from the 'baseline1' codebase, run 'in Claude Code with Claude Opus 5 as the backbone model and reasoning effort set to extra-high'; the agent sees only the action interface, not the environment source - Result: all 25 public games cleared, mean RHAE (Relative Human Action Efficiency) 100.0, 7,518 actions against a human baseline of 17,135 (0.44); per-game ratios from 0.20 to 0.67 - Comparison in the paper: 59.3 points higher mean RHAE than Claude Opus 5 with the ARC Prize standard harness (which implies 40.7) - The gain over the best earlier harness is small: 'baseline1' also finishes every game, with 8,347 actions (0.49 of the human count) and a mean RHAE of 99.0 in the paper's table; the other four community-leaderboard systems compared (OPINE-World, NOOA, DreamTeam, Continual Harness) leave 5, 6, 19 and 22 games unfinished (mean RHAE 78.4, 85.1, 38.1 and 20.5) - Ablation on three games: the rulebook cuts actions by 9% (617 vs 677) and agent turns by 18% (678 vs 830); a two-model population on game wa30 cuts actions from 899 to 597 - Atari Pong case study: a learned feedback controller wins 21:0 in three evaluated episodes without LLM calls during play - Caveat stated by the authors: 'Public-set completion alone therefore cannot isolate the contribution of a world-model architecture'; comparisons must also account for model choice, reasoning budget and held-out evaluation. No semi-private or ARC Prize-verified score is reported, and the paper gives no cost figure beyond 'substantial time and monetary cost' ## What happened Memento 3 continues the Memento series of agents that learn through external memory instead of weight updates. Earlier versions stored experiences and reusable skills; this one learns the "model side", the rules of an unfamiliar environment. The agent explores a game, writes what it believes about the dynamics into a Markdown rulebook, compiles the rulebook into a Python world model, plans with that model, and repairs both when a prediction fails. The authors present this as a model-based route to recursive self-improvement in which "the underlying LLM remains fixed". The numbers come from the paper (read from the PDF). Every figure for other systems is taken by the authors from those systems' entries on the ARC-AGI-3 community leaderboard. ## Why it matters ARC-AGI-3's public games are now solved at the efficiency ceiling by two harnesses built on the same frozen model, while that model alone scores about 41 with the standard harness. The difference comes from the scaffolding, which is the same pattern as in other recent harness results. The result does not show performance on held-out games, which is what the benchmark's official scores measure. ## Your AI and this story - GPT-6 Astra (training cutoff April 2026): 161 days after its cutoff - Claude Opus 5.5 (training cutoff June 2026): 100 days after its cutoff - Gemini 3.8 Flash (training cutoff March 2026): 191 days after its cutoff - Grok 4.7 (training cutoff May 2026): 130 days after its cutoff ## Sources 1. [arXiv 2610.11794: Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks](https://arxiv.org/abs/2610.11794) (arxiv.org, paper) 2. [Pandaily: Huawei Noah's Ark Lab's Memento 3 Agent Clears All 25 ARC-AGI-3 Public Games With a Frozen Model (headline only; page text did not load)](https://pandaily.com/huawei-noahs-ark-lab-memento-3-arc-agi-3-public-set-code-world-model-frozen-claude) (pandaily.com, press) 3. [AI Weekly: Memento 3 Posts 100.0 RHAE on ARC-AGI-3 With Frozen LLM Rulebook](https://aiweekly.co/alerts/memento-3-posts-1000-rhae-on-arc-agi-3-with-frozen-llm-rulebook) (aiweekly.co, press) ## Changes - 2026-10-12 (filed): Created (Pandaily Oct 12) ## Related - 2026-10-01: [Open-source ARC-AGI-3 agents top out at 27.9% at ARC Prize 2026 Milestone #2](https://postcutoff.com/e/2026-10-01-arc-prize-2026-milestone-2-winners/index.md) - 2026-09-28: [Harness learning: a small model trained with RL learns to rewrite an agent's harness from execution feedback](https://postcutoff.com/e/2026-09-28-harness-learning-test-time-adaptation/index.md) - 2026-09-03: [GPT-6 Astra scores 62.7% on ARC-AGI-3, outacting humans on 96% of levels](https://postcutoff.com/e/2026-09-03-arc-agi-3-gpt-6-astra/index.md) - 2026-07-24: [Anthropic releases Claude Opus 5](https://postcutoff.com/e/2026-07-24-claude-opus-5/index.md) - 2026-03-25: [ARC Prize launches ARC-AGI-3, an interactive game benchmark where frontier AI scored under 1%](https://postcutoff.com/e/2026-03-25-arc-agi-3-launch/index.md)