---
id: "2017-06-12-transformer"
url: "https://postcutoff.com/e/2017-06-12-transformer/"
as_of: "2026-11-13T13:49:00+01:00"
date: "2017-06-12"
date_precision: day
category: research
importance: 5
confidence: high
status: [Confirmed]
sources: 3
editor: human editor
human_review: null
version: "2026-10-04"
---

As of: 2026-11-13 13:49 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2017-06-12-transformer/

# 'Attention Is All You Need' introduces the Transformer

Vaswani et al. proposed the Transformer, an architecture built entirely on self-attention without recurrence; it became the foundation of BERT, GPT and virtually every modern large AI model.

## Key facts
- arXiv 1706.03762, posted 12 June 2017; NeurIPS 2017
- Eight co-authors from Google Brain/Research
- WMT 2014 English–German: 28.4 BLEU, a new state of the art
- Highly parallelizable training vs. RNNs, enabling scale
- The 'T' in GPT stands for Transformer

## What happened

The paper introduced multi-head self-attention, positional encodings and an encoder–decoder stack, beating recurrent models on translation while training much faster.

## Why it matters

Arguably the most consequential AI paper of the century so far: the Transformer's scalability made LLMs, multimodal models and AlphaFold 2 possible.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): in its training data
- Claude Opus 5.5 (training cutoff June 2026): in its training data
- Gemini 4 Hydrogen (training cutoff March 2026): in its training data
- Grok 4.7 (training cutoff May 2026): in its training data

## Sources

1. [Attention Is All You Need (arXiv)](https://arxiv.org/abs/1706.03762) (arxiv.org, paper)
2. [Google Research blog: Transformer](https://research.google/blog/transformer-a-novel-neural-network-architecture-for-language-understanding/) (research.google, official)
3. [Wikipedia: Attention Is All You Need](https://en.wikipedia.org/wiki/Attention_Is_All_You_Need) (en.wikipedia.org, discussion)

## Changes

- 2026-09-29 (filed): Created

## Related

- 2018-10-11: [Google releases BERT, bidirectional Transformer pre-training](https://postcutoff.com/e/2018-10-11-bert/index.md)
- 2018-06-11: [OpenAI's GPT-1: generative pre-training of Transformers](https://postcutoff.com/e/2018-06-11-gpt-1/index.md)
- 2014-09-10: [Sequence-to-sequence learning and neural attention](https://postcutoff.com/e/2014-09-10-seq2seq-attention/index.md)