{"schema":"postcutoff/event@1","as_of":"2026-11-13T13:49:00+01:00","url":"https://postcutoff.com/e/2014-09-10-seq2seq-attention/","md":"https://postcutoff.com/e/2014-09-10-seq2seq-attention/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2014-09-10-seq2seq-attention","date":"2014-09-10","date_precision":"day","short_title":"Sequence-to-sequence learning and neural attention","deck":null,"takeaway":"Sutskever, Vinyals and Le's seq2seq (LSTM encoder–decoder) and Bahdanau, Cho and Bengio's attention mechanism, both posted in September 2014, made end-to-end neural machine translation work.","category":"research","category_label":"Research","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"Sequence to Sequence Learning with Neural Networks (arXiv)","url":"https://arxiv.org/abs/1409.3215","type":"paper","group":"primary","domain":"arxiv.org"},{"n":2,"title":"Neural Machine Translation by Jointly Learning to Align and Translate (arXiv)","url":"https://arxiv.org/abs/1409.0473","type":"paper","group":"primary","domain":"arxiv.org"},{"n":3,"title":"Google's Neural Machine Translation System (arXiv)","url":"https://arxiv.org/abs/1609.08144","type":"paper","group":"primary","domain":"arxiv.org"}],"official":3,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Google","Université de Montréal"],"title":"Sequence-to-sequence learning and neural attention","summary":"Sutskever, Vinyals and Le's seq2seq (LSTM encoder–decoder) and Bahdanau, Cho and Bengio's attention mechanism, both posted in September 2014, made end-to-end neural machine translation work.","key_facts":["Bahdanau et al. attention paper: arXiv 1409.0473 (1 Sep 2014)","Sutskever et al. seq2seq paper: arXiv 1409.3215 (10 Sep 2014)","Google Neural Machine Translation system launched in 2016 (arXiv 1609.08144)","Attention later became the sole core mechanism of the Transformer"],"key_numbers":[],"tags":["nlp","machine-translation","attention","rnn"],"science":null,"body_md":"## What happened\n\nTwo papers showed neural networks could map whole sequences to sequences, and that letting the decoder 'attend' to encoder states greatly improved long sentences.\n\n## Why it matters\n\nEstablished the encoder–decoder paradigm and attention — the direct precursors of the Transformer and modern LLMs.","disputed":[],"related":[{"id":"2017-06-12-transformer","url":"https://postcutoff.com/e/2017-06-12-transformer/","date":"2017-06-12","date_precision":"day","short_title":"'Attention Is All You Need' introduces the Transformer","deck":null,"takeaway":"Arguably the most consequential AI paper of the century so far: the Transformer's scalability made LLMs, multimodal models and AlphaFold 2 possible.","category":"research","category_label":"Research","importance":5,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":3,"official":2,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Google Brain","Google Research"]},{"id":"1997-11-15-lstm","url":"https://postcutoff.com/e/1997-11-15-lstm/","date":"1997-11-15","date_precision":"month","short_title":"Hochreiter & Schmidhuber introduce Long Short-Term Memory (LSTM)","deck":null,"takeaway":"LSTMs dominated sequence modeling (speech, translation, handwriting) until the Transformer, and underpinned early seq2seq systems like Google Translate's 2016 neural system.","category":"research","category_label":"Research","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":2,"official":1,"filed":"2026-09-29","updated":"2026-09-29","orgs":["TU Munich","IDSIA"]}],"people":[{"id":"ilya-sutskever","name":"Ilya Sutskever","url":"https://postcutoff.com/person/ilya-sutskever/"},{"id":"yoshua-bengio","name":"Yoshua Bengio","url":"https://postcutoff.com/person/yoshua-bengio/"},{"id":"oriol-vinyals","name":"Oriol Vinyals","url":"https://postcutoff.com/person/oriol-vinyals/"}],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-09-29","type":"filed","text":"Created"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-29","run":null,"sources_read":null,"updated":"2026-09-29","human_review":null,"version":{"date":"2026-10-04"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":null,"in_training_data":true},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":null,"in_training_data":true},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":null,"in_training_data":true},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":null,"in_training_data":true}],"short_url":"https://postcutoff.com/s/sequence-to-sequence-learning"}