Sequence-to-sequence learning and neural attention
Confirmed
Importance: major (4 of 5)The takeaway
Sutskever, Vinyals and Le’s seq2seq (LSTM encoder–decoder) and Bahdanau, Cho and Bengio’s attention mechanism, both posted in September 2014, made end-to-end neural machine translation work.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- Major (4 of 5)
- Last verified
- 29 September 2026
Your AI and this story
- GPT-6 AstraIn its training data
- Claude Opus 5.5In its training data
- Gemini 4 HydrogenIn its training data
- Grok 4.7In its training data
It happened before the cutoffs of all four assistants, so it can be in their training data.
Key facts
- Bahdanau et al. attention paper: arXiv 1409.0473 (1 Sep 2014)
- Sutskever et al. seq2seq paper: arXiv 1409.3215 (10 Sep 2014)
- Google Neural Machine Translation system launched in 2016 (arXiv 1609.08144)
- Attention later became the sole core mechanism of the Transformer
What happened
Two papers showed neural networks could map whole sequences to sequences, and that letting the decoder ‘attend’ to encoder states greatly improved long sentences.
Why it matters
Established the encoder–decoder paradigm and attention — the direct precursors of the Transformer and modern LLMs.
Sources
3 sources from 1 site. Numbers match the chips in the text.
3 sources: 3 primary
Primary
- Sequence to Sequence Learning with Neural Networks (arXiv)arxiv.org, paper
- Neural Machine Translation by Jointly Learning to Align and Translate (arXiv)arxiv.org, paper
- Google’s Neural Machine Translation System (arXiv)arxiv.org, paper
Changes
- Filed