--- id: "2026-10-01-nat-arc-natural-image-pretraining-arc" url: "https://postcutoff.com/e/2026-10-01-nat-arc-natural-image-pretraining-arc/" as_of: "2026-11-22T13:32:00+01:00" date: "2026-10-01" date_precision: day category: research importance: 2 confidence: high status: [Confirmed] sources: 4 editor: human editor human_review: null version: "2026-10-07" --- As of: 2026-11-22 13:32 CET. Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/ Canonical page: https://postcutoff.com/e/2026-10-01-nat-arc-natural-image-pretraining-arc/ # Kaiming He's MIT group: ImageNet pretraining lifts a pure-vision ARC solver to 63.4% Full title: Kaiming He's MIT group: ImageNet pretraining lifts a pure-vision ARC solver (Nat-ARC) to 63.4% on ARC-AGI-1, 70.2% ensembled "Natural Image Pretraining Improves Abstract Reasoning" (Ding, Hu, Gan, Yin, Kaiming He; MIT; ECCV 2026) introduces Nat-ARC. It extends the vision-only VARC pipeline by initializing its ViT encoder from Masked Autoencoders pretrained on ImageNet. The best single model reaches 63.4 ± 0.7% pass@2 on ARC-1, and an ensemble reaches 70.2 ± 0.6%, with no language model. Chinese media covered it on Oct 1, 2026. ## Key facts - Best ImageNet-MAE-pretrained single model (huge, ~0.6B parameters per QbitAI): 63.4 ± 0.7% pass@2 on ARC-1 - Prediction-level ensemble of no pretraining, ImageNet MAE pretraining and ARC-style grid pretraining: 70.2 ± 0.6% pass@2 on ARC-1, 'using a purely vision-based method' - ImageNet MAE pretraining improves over matched randomly initialized baselines at base, large and huge scales - Predecessor: VARC ('ARC Is a Vision Problem!', arXiv 2511.14761, Nov 2025), 60.4% on ARC-1 with an 18M-parameter ViT trained from scratch - Tasks helped most involve visual matching, copying and connected-component reasoning (authors' qualitative analysis) - QbitAI's comparison: LoopViT 65.8%, Loop-OWM 68.5%, and the LLM-based ARChitects (8B) 71.6% (as reported by QbitAI, not checked against the paper's tables) ## What happened Kaiming He's group at MIT asked whether visual knowledge from natural photos transfers to ARC's abstract colored-grid puzzles. They took the VARC pipeline, which treats each ARC task as image-to-image translation with offline training plus per-task test-time training. They replaced its random initialization with an encoder pretrained as a Masked Autoencoder on ImageNet. Pretraining helped at every model scale, and ensembling differently pretrained models reached 70.2% pass@2 on ARC-1 without any language model. ## Why it matters Almost all high ARC-AGI-1 scores come from LLM pipelines. Nat-ARC shows that a small vision-only model with generic visual pretraining gets close, which supports the view that much of ARC is perception plus few-shot adaptation. ARC-1 is now largely saturated by frontier LLMs, and the paper reports no ARC-AGI-2 or ARC-AGI-3 results. ## Your AI and this story - GPT-6.2 Astra (training cutoff May 2026): 123 days after its cutoff - Claude Opus 5.5 (training cutoff June 2026): 93 days after its cutoff - Gemini 4 Hydrogen (training cutoff March 2026): 184 days after its cutoff - Grok 4.7 (training cutoff May 2026): 123 days after its cutoff ## Sources 1. [ECCV 2026 poster: Natural Image Pretraining Improves Abstract Reasoning](https://eccv.ecva.net/virtual/2026/poster/5527) (eccv.ecva.net, paper) 2. [Paper PDF (ECCV 2026)](https://media.eventhosts.cc/Conferences/ECCV2026/pdfs/11149.pdf) (media.eventhosts.cc, paper) 3. [arXiv 2511.14761: ARC Is a Vision Problem! (VARC)](https://arxiv.org/abs/2511.14761) (arxiv.org, paper) 4. [QbitAI: Kaiming He's team learns ARC from cat pictures (Chinese)](https://www.qbitai.com/2026/10/499812.html) (qbitai.com, press) ## Changes - 2026-10-02 (filed): Created (QbitAI) ## Related - 2026-03-25: [ARC Prize launches ARC-AGI-3, an interactive game benchmark where frontier AI scored under 1%](https://postcutoff.com/e/2026-03-25-arc-agi-3-launch/index.md)