{"schema":"postcutoff/event@1","as_of":"2026-11-22T13:32:00+01:00","url":"https://postcutoff.com/e/2026-09-30-wild-ai-web-text-scaling-laws/","md":"https://postcutoff.com/e/2026-09-30-wild-ai-web-text-scaling-laws/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-09-30-wild-ai-web-text-scaling-laws","date":"2026-09-30","date_precision":"day","short_title":"Study: ~31% of filtered web text was AI-generated by Aug 2026, and it hurts pretraining","deck":null,"takeaway":"A Sept 30, 2026 arXiv paper (Russell, Iyyer, Spero, Emi et al.) measured that 27.5% of FineWeb-filtered tokens in June 2026 web data were AI-generated (by Pangram's detector), rising to 31.1% in August.","category":"research","category_label":"Research","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"arXiv 2609.40295: How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text","url":"https://arxiv.org/abs/2609.40295","type":"paper","group":"primary","domain":"arxiv.org"},{"n":2,"title":"GitHub: pangramlabs/WildAI","url":"https://github.com/pangramlabs/WildAI","type":"code","group":"primary","domain":"github.com"},{"n":3,"title":"Hugging Face: pangram/WildAI dataset","url":"https://huggingface.co/datasets/pangram/WildAI","type":"code","group":"primary","domain":"huggingface.co"},{"n":4,"title":"Hugging Face: pangram/WildAI-models","url":"https://huggingface.co/pangram/WildAI-models","type":"code","group":"primary","domain":"huggingface.co"},{"n":5,"title":"Ken Ashe: Wild AI web text is starting to poison pretraining","url":"https://kenashe.ai/blog/2026-10-01-wild-ai-web-text-is-starting-to-poison-pretraining","type":"discussion","group":"reactions","domain":"kenashe.ai"}],"official":4,"filed":"2026-10-03","updated":"2026-10-03","orgs":["Pangram Labs","UMass Amherst"],"title":"Pangram/UMass study: ~31% of filtered web tokens were AI-generated by Aug 2026, and such 'wild' AI text soon hurts pretraining","summary":"A Sept 30, 2026 arXiv paper (Russell, Iyyer, Spero, Emi et al.) measured that 27.5% of FineWeb-filtered tokens in June 2026 web data were AI-generated (by Pangram's detector), rising to 31.1% in August. Pretraining 800 models, it found that such \"wild\" AI text helps data-starved models a little but quickly turns harmful, and that Chinchilla-style scaling laws fail to predict this. It proposes a new law and releases the 83B-token WildAI corpus.","key_facts":["AI share of FineWeb-filtered web tokens (Pangram classifier): 27.5% in June 2026 → 31.1% in August 2026","800 language models pretrained with varying ratios of added AI tokens to human tokens","Data-starved models: AI tokens first lower loss on human text, then the benefit saturates and reverses into harm; models with large human budgets: AI tokens raise loss almost immediately","Training on unfiltered web text at August 2026's AI share needs about 1.6x the compute of training on its human subset (at 20 tokens per parameter), a gap that grows with the human budget","New scaling law with separate benefit and harm terms reduces to Chinchilla without AI text; fit on small models, it predicts models up to 3.6x larger with 41% lower error than the best existing law","Released: WildAI corpus (83B tokens, with AI/topic/format labels), the 800 models and code (CC BY-NC-SA 4.0)","Caveat: AI-share numbers depend on Pangram's own detector; Pangram co-founders Max Spero and Bradley Emi are authors"],"key_numbers":[],"tags":["pretraining","data","model-collapse","scaling-laws","ai-generated-text","web-data"],"science":null,"body_md":"## What happened\n\nThe authors labelled recent Common Crawl data after FineWeb quality filtering with Pangram's AI-text detector, then ran a large controlled pretraining study that mixed in AI-written web text (not synthetic data from one model, but text from many models written for human readers). They fitted scaling laws to held-out loss on human and on AI text.\n\n## Why it matters\n\nIt is a quantitative estimate that almost a third of quality-filtered web text is now machine-written, and evidence that this text has negative value for well-resourced pretraining. Pre-2023 human text and data filtering become more valuable, and labs that cannot detect AI text pay a compute penalty.","disputed":[],"related":[{"id":"2026-08-26-semafor-ai-written-op-eds-pangram","url":"https://postcutoff.com/e/2026-08-26-semafor-ai-written-op-eds-pangram/","date":"2026-08-26","date_precision":"day","short_title":"Semafor finds AI-written guest op-eds in the NYT, WSJ and Washington Post","deck":"Dartmouth provost's case becomes a campus scandal","takeaway":"This is a concrete measurement of undisclosed AI ghostwriting in elite public discourse, and an early case of senior institutional leaders being held to the AI-disclosure rules they impose on students.","category":"culture","category_label":"Culture","importance":2,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":5,"official":0,"filed":"2026-10-02","updated":"2026-10-02","orgs":["Semafor","Pangram Labs","Dartmouth College"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-10-03","type":"filed","text":"Created (Oct 2)"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-10-03","run":null,"sources_read":null,"updated":"2026-10-03","human_review":null,"version":{"date":"2026-10-07"}},"gaps":[{"model_id":"gpt-6-2-astra","name":"GPT-6.2 Astra","cutoff":"2026-05","days_after":122,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":92,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":183,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":122,"in_training_data":false}],"short_url":null}