{"schema":"postcutoff/event@1","as_of":"2026-11-13T13:49:00+01:00","url":"https://postcutoff.com/e/2018-06-11-gpt-1/","md":"https://postcutoff.com/e/2018-06-11-gpt-1/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2018-06-11-gpt-1","date":"2018-06-11","date_precision":"day","short_title":"OpenAI's GPT-1: generative pre-training of Transformers","deck":null,"takeaway":"OpenAI showed that pre-training a Transformer language model on unlabeled text and then fine-tuning it yields strong results across many NLP tasks — the first 'GPT'.","category":"model-release","category_label":"Model releases","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"Improving language understanding with unsupervised learning (OpenAI)","url":"https://openai.com/index/language-unsupervised/","type":"official","group":"primary","domain":"openai.com"},{"n":2,"title":"Paper PDF","url":"https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf","type":"paper","group":"primary","domain":"cdn.openai.com"}],"official":2,"filed":"2026-09-29","updated":"2026-09-29","orgs":["OpenAI"],"title":"OpenAI's GPT-1: generative pre-training of Transformers","summary":"OpenAI showed that pre-training a Transformer language model on unlabeled text and then fine-tuning it yields strong results across many NLP tasks — the first 'GPT'.","key_facts":["Paper: 'Improving Language Understanding by Generative Pre-Training' (Radford et al.)","~117M parameters, 12-layer decoder-only Transformer","Pre-trained on the BooksCorpus dataset","Improved state of the art on 9 of 12 benchmarks studied"],"key_numbers":[],"tags":["llm","gpt","pretraining","transformer"],"science":null,"body_md":"## What happened\n\nOpenAI published a semi-supervised approach: unsupervised generative pre-training followed by supervised fine-tuning.\n\n## Why it matters\n\nEstablished the pre-train-then-adapt paradigm and the decoder-only Transformer lineage that led to ChatGPT.","disputed":[],"related":[{"id":"2019-02-14-gpt-2","url":"https://postcutoff.com/e/2019-02-14-gpt-2/","date":"2019-02-14","date_precision":"day","short_title":"OpenAI announces GPT-2 and withholds the full model over misuse concerns","deck":null,"takeaway":"First public glimpse of what scaling LLMs could do and the first major debate over whether to release model weights.","category":"model-release","category_label":"Model releases","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":3,"official":3,"filed":"2026-09-29","updated":"2026-09-29","orgs":["OpenAI"]},{"id":"2017-06-12-transformer","url":"https://postcutoff.com/e/2017-06-12-transformer/","date":"2017-06-12","date_precision":"day","short_title":"'Attention Is All You Need' introduces the Transformer","deck":null,"takeaway":"Arguably the most consequential AI paper of the century so far: the Transformer's scalability made LLMs, multimodal models and AlphaFold 2 possible.","category":"research","category_label":"Research","importance":5,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":3,"official":2,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Google Brain","Google Research"]}],"people":[{"id":"alec-radford","name":"Alec Radford","url":"https://postcutoff.com/person/alec-radford/"}],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-09-29","type":"filed","text":"Created"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-29","run":null,"sources_read":null,"updated":"2026-09-29","human_review":null,"version":{"date":"2026-10-04"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":null,"in_training_data":true},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":null,"in_training_data":true},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":null,"in_training_data":true},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":null,"in_training_data":true}],"short_url":"https://postcutoff.com/s/openai-gpt-1-generative"}