OpenAI publishes ‘Scaling Laws for Neural Language Models’
Confirmed
Importance: historic (5 of 5)The takeaway
Kaplan et al. showed language-model loss falls as a smooth power law in parameters, data and compute over many orders of magnitude, giving a quantitative case for building ever-larger models.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- Historic (5 of 5)
- Last verified
- 29 September 2026
Your AI and this story
- GPT-6 AstraIn its training data
- Claude Opus 5.5In its training data
- Gemini 4 HydrogenIn its training data
- Grok 4.7In its training data
It happened before the cutoffs of all four assistants, so it can be in their training data.
Key facts
- arXiv 2001.08361 (January 2020)
- Loss follows power laws in model size, dataset size and compute
- Architecture details (depth/width) matter far less than scale
- Later revised by DeepMind’s Chinchilla (2022) on the optimal data/parameter ratio
What happened
The paper fit empirical power laws across hundreds of training runs and derived compute-optimal allocation rules.
Why it matters
Scaling laws became the strategic basis for the trillion-dollar compute build-out of the 2020s.
Sources
2 sources from 2 sites. Numbers match the chips in the text.
2 sources: 1 primary, 1 reaction
Primary
- Scaling Laws for Neural Language Models (arXiv)arxiv.org, paper
Reactions
- Wikipedia: Neural scaling lawen.wikipedia.org, discussion
Changes
- Filed