---
id: "2014-09-10-seq2seq-attention"
url: "https://postcutoff.com/e/2014-09-10-seq2seq-attention/"
as_of: "2026-11-13T13:49:00+01:00"
date: "2014-09-10"
date_precision: day
category: research
importance: 4
confidence: high
status: [Confirmed]
sources: 3
editor: human editor
human_review: null
version: "2026-10-04"
---

As of: 2026-11-13 13:49 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2014-09-10-seq2seq-attention/

# Sequence-to-sequence learning and neural attention

Sutskever, Vinyals and Le's seq2seq (LSTM encoder–decoder) and Bahdanau, Cho and Bengio's attention mechanism, both posted in September 2014, made end-to-end neural machine translation work.

## Key facts
- Bahdanau et al. attention paper: arXiv 1409.0473 (1 Sep 2014)
- Sutskever et al. seq2seq paper: arXiv 1409.3215 (10 Sep 2014)
- Google Neural Machine Translation system launched in 2016 (arXiv 1609.08144)
- Attention later became the sole core mechanism of the Transformer

## What happened

Two papers showed neural networks could map whole sequences to sequences, and that letting the decoder 'attend' to encoder states greatly improved long sentences.

## Why it matters

Established the encoder–decoder paradigm and attention — the direct precursors of the Transformer and modern LLMs.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): in its training data
- Claude Opus 5.5 (training cutoff June 2026): in its training data
- Gemini 4 Hydrogen (training cutoff March 2026): in its training data
- Grok 4.7 (training cutoff May 2026): in its training data

## Sources

1. [Sequence to Sequence Learning with Neural Networks (arXiv)](https://arxiv.org/abs/1409.3215) (arxiv.org, paper)
2. [Neural Machine Translation by Jointly Learning to Align and Translate (arXiv)](https://arxiv.org/abs/1409.0473) (arxiv.org, paper)
3. [Google's Neural Machine Translation System (arXiv)](https://arxiv.org/abs/1609.08144) (arxiv.org, paper)

## Changes

- 2026-09-29 (filed): Created

## Related

- 2017-06-12: ['Attention Is All You Need' introduces the Transformer](https://postcutoff.com/e/2017-06-12-transformer/index.md)
- 1997-11-15: [Hochreiter & Schmidhuber introduce Long Short-Term Memory (LSTM)](https://postcutoff.com/e/1997-11-15-lstm/index.md)
- People: [Ilya Sutskever](https://postcutoff.com/person/ilya-sutskever/), [Yoshua Bengio](https://postcutoff.com/person/yoshua-bengio/), [Oriol Vinyals](https://postcutoff.com/person/oriol-vinyals/)