‘Attention Is All You Need’ introduces the Transformer
Confirmed
Importance: historic (5 of 5)The takeaway
Vaswani et al. proposed the Transformer, an architecture built entirely on self-attention without recurrence; it became the foundation of BERT, GPT and virtually every modern large AI model.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- Historic (5 of 5)
- Last verified
- 29 September 2026
Your AI and this story
- GPT-6 AstraIn its training data
- Claude Opus 5.5In its training data
- Gemini 4 HydrogenIn its training data
- Grok 4.7In its training data
It happened before the cutoffs of all four assistants, so it can be in their training data.
Key facts
- arXiv 1706.03762, posted 12 June 2017; NeurIPS 2017
- Eight co-authors from Google Brain/Research
- WMT 2014 English–German: 28.4 BLEU, a new state of the art
- Highly parallelizable training vs. RNNs, enabling scale
- The ‘T’ in GPT stands for Transformer
What happened
The paper introduced multi-head self-attention, positional encodings and an encoder–decoder stack, beating recurrent models on translation while training much faster.
Why it matters
Arguably the most consequential AI paper of the century so far: the Transformer’s scalability made LLMs, multimodal models and AlphaFold 2 possible.
Sources
3 sources from 3 sites. Numbers match the chips in the text.
3 sources: 2 primary, 1 reaction
Primary
- Attention Is All You Need (arXiv)arxiv.org, paper
- Google Research blog: Transformerresearch.google, official
Reactions
- Wikipedia: Attention Is All You Needen.wikipedia.org, discussion
Changes
- Filed