---
id: "2022-03-29-chinchilla"
url: "https://postcutoff.com/e/2022-03-29-chinchilla/"
as_of: "2026-11-13T13:49:00+01:00"
date: "2022-03-29"
date_precision: day
category: research
importance: 4
confidence: high
status: [Confirmed]
sources: 2
editor: human editor
human_review: null
version: "2026-10-04"
---

As of: 2026-11-13 13:49 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2022-03-29-chinchilla/

# DeepMind's Chinchilla revises scaling laws toward more data

Hoffmann et al. found that for compute-optimal training, parameters and training tokens should scale equally (~20 tokens per parameter); 70B Chinchilla outperformed the 280B Gopher.

## Key facts
- arXiv 2203.15556 'Training Compute-Optimal Large Language Models'
- Chinchilla: 70B parameters trained on 1.4 trillion tokens
- Beat Gopher (280B), GPT-3 (175B) and Megatron-Turing NLG (530B) on many benchmarks
- Implied most prior LLMs were undertrained

## What happened

Over 400 training runs showed earlier scaling laws had over-weighted parameter count relative to data.

## Why it matters

Reshaped how every lab trains LLMs, pushing toward far larger datasets and smaller, cheaper-to-serve models (e.g. LLaMA).

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): in its training data
- Claude Opus 5.5 (training cutoff June 2026): in its training data
- Gemini 4 Hydrogen (training cutoff March 2026): in its training data
- Grok 4.7 (training cutoff May 2026): in its training data

## Sources

1. [Training Compute-Optimal Large Language Models (arXiv)](https://arxiv.org/abs/2203.15556) (arxiv.org, paper)
2. [Wikipedia: Chinchilla (language model)](https://en.wikipedia.org/wiki/Chinchilla_(language_model)) (en.wikipedia.org, discussion)

## Changes

- 2026-09-29 (filed): Created

## Related

- 2023-02-24: [Meta releases LLaMA, sparking the open-weights LLM wave](https://postcutoff.com/e/2023-02-24-llama/index.md)
- 2020-01-23: [OpenAI publishes 'Scaling Laws for Neural Language Models'](https://postcutoff.com/e/2020-01-23-scaling-laws/index.md)