---
id: "2020-01-23-scaling-laws"
url: "https://postcutoff.com/e/2020-01-23-scaling-laws/"
as_of: "2026-11-13T13:49:00+01:00"
date: "2020-01-23"
date_precision: day
category: research
importance: 5
confidence: high
status: [Confirmed]
sources: 2
editor: human editor
human_review: null
version: "2026-10-04"
---

As of: 2026-11-13 13:49 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2020-01-23-scaling-laws/

# OpenAI publishes 'Scaling Laws for Neural Language Models'

Kaplan et al. showed language-model loss falls as a smooth power law in parameters, data and compute over many orders of magnitude, giving a quantitative case for building ever-larger models.

## Key facts
- arXiv 2001.08361 (January 2020)
- Loss follows power laws in model size, dataset size and compute
- Architecture details (depth/width) matter far less than scale
- Later revised by DeepMind's Chinchilla (2022) on the optimal data/parameter ratio

## What happened

The paper fit empirical power laws across hundreds of training runs and derived compute-optimal allocation rules.

## Why it matters

Scaling laws became the strategic basis for the trillion-dollar compute build-out of the 2020s.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): in its training data
- Claude Opus 5.5 (training cutoff June 2026): in its training data
- Gemini 4 Hydrogen (training cutoff March 2026): in its training data
- Grok 4.7 (training cutoff May 2026): in its training data

## Sources

1. [Scaling Laws for Neural Language Models (arXiv)](https://arxiv.org/abs/2001.08361) (arxiv.org, paper)
2. [Wikipedia: Neural scaling law](https://en.wikipedia.org/wiki/Neural_scaling_law) (en.wikipedia.org, discussion)

## Changes

- 2026-09-29 (filed): Created

## Related

- 2022-03-29: [DeepMind's Chinchilla revises scaling laws toward more data](https://postcutoff.com/e/2022-03-29-chinchilla/index.md)
- 2020-05-28: [GPT-3 (175B) shows in-context few-shot learning](https://postcutoff.com/e/2020-05-28-gpt-3/index.md)