--- id: "2026-09-30-wild-ai-web-text-scaling-laws" url: "https://postcutoff.com/e/2026-09-30-wild-ai-web-text-scaling-laws/" as_of: "2026-11-22T13:32:00+01:00" date: "2026-09-30" date_precision: day category: research importance: 3 confidence: high status: [Confirmed] sources: 5 editor: human editor human_review: null version: "2026-10-07" --- As of: 2026-11-22 13:32 CET. Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/ Canonical page: https://postcutoff.com/e/2026-09-30-wild-ai-web-text-scaling-laws/ # Study: ~31% of filtered web text was AI-generated by Aug 2026, and it hurts pretraining Full title: Pangram/UMass study: ~31% of filtered web tokens were AI-generated by Aug 2026, and such 'wild' AI text soon hurts pretraining A Sept 30, 2026 arXiv paper (Russell, Iyyer, Spero, Emi et al.) measured that 27.5% of FineWeb-filtered tokens in June 2026 web data were AI-generated (by Pangram's detector), rising to 31.1% in August. Pretraining 800 models, it found that such "wild" AI text helps data-starved models a little but quickly turns harmful, and that Chinchilla-style scaling laws fail to predict this. It proposes a new law and releases the 83B-token WildAI corpus. ## Key facts - AI share of FineWeb-filtered web tokens (Pangram classifier): 27.5% in June 2026 → 31.1% in August 2026 - 800 language models pretrained with varying ratios of added AI tokens to human tokens - Data-starved models: AI tokens first lower loss on human text, then the benefit saturates and reverses into harm; models with large human budgets: AI tokens raise loss almost immediately - Training on unfiltered web text at August 2026's AI share needs about 1.6x the compute of training on its human subset (at 20 tokens per parameter), a gap that grows with the human budget - New scaling law with separate benefit and harm terms reduces to Chinchilla without AI text; fit on small models, it predicts models up to 3.6x larger with 41% lower error than the best existing law - Released: WildAI corpus (83B tokens, with AI/topic/format labels), the 800 models and code (CC BY-NC-SA 4.0) - Caveat: AI-share numbers depend on Pangram's own detector; Pangram co-founders Max Spero and Bradley Emi are authors ## What happened The authors labelled recent Common Crawl data after FineWeb quality filtering with Pangram's AI-text detector, then ran a large controlled pretraining study that mixed in AI-written web text (not synthetic data from one model, but text from many models written for human readers). They fitted scaling laws to held-out loss on human and on AI text. ## Why it matters It is a quantitative estimate that almost a third of quality-filtered web text is now machine-written, and evidence that this text has negative value for well-resourced pretraining. Pre-2023 human text and data filtering become more valuable, and labs that cannot detect AI text pay a compute penalty. ## Your AI and this story - GPT-6.2 Astra (training cutoff May 2026): 122 days after its cutoff - Claude Opus 5.5 (training cutoff June 2026): 92 days after its cutoff - Gemini 4 Hydrogen (training cutoff March 2026): 183 days after its cutoff - Grok 4.7 (training cutoff May 2026): 122 days after its cutoff ## Sources 1. [arXiv 2609.40295: How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text](https://arxiv.org/abs/2609.40295) (arxiv.org, paper) 2. [GitHub: pangramlabs/WildAI](https://github.com/pangramlabs/WildAI) (github.com, code) 3. [Hugging Face: pangram/WildAI dataset](https://huggingface.co/datasets/pangram/WildAI) (huggingface.co, code) 4. [Hugging Face: pangram/WildAI-models](https://huggingface.co/pangram/WildAI-models) (huggingface.co, code) 5. [Ken Ashe: Wild AI web text is starting to poison pretraining](https://kenashe.ai/blog/2026-10-01-wild-ai-web-text-is-starting-to-poison-pretraining) (kenashe.ai, discussion) ## Changes - 2026-10-03 (filed): Created (Oct 2) ## Related - 2026-08-26: [Semafor finds AI-written guest op-eds in the NYT, WSJ and Washington Post](https://postcutoff.com/e/2026-08-26-semafor-ai-written-op-eds-pangram/index.md)