---
id: "2018-06-11-gpt-1"
url: "https://postcutoff.com/e/2018-06-11-gpt-1/"
as_of: "2026-11-13T13:49:00+01:00"
date: "2018-06-11"
date_precision: day
category: model-release
importance: 4
confidence: high
status: [Confirmed]
sources: 2
editor: human editor
human_review: null
version: "2026-10-04"
---

As of: 2026-11-13 13:49 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2018-06-11-gpt-1/

# OpenAI's GPT-1: generative pre-training of Transformers

OpenAI showed that pre-training a Transformer language model on unlabeled text and then fine-tuning it yields strong results across many NLP tasks — the first 'GPT'.

## Key facts
- Paper: 'Improving Language Understanding by Generative Pre-Training' (Radford et al.)
- ~117M parameters, 12-layer decoder-only Transformer
- Pre-trained on the BooksCorpus dataset
- Improved state of the art on 9 of 12 benchmarks studied

## What happened

OpenAI published a semi-supervised approach: unsupervised generative pre-training followed by supervised fine-tuning.

## Why it matters

Established the pre-train-then-adapt paradigm and the decoder-only Transformer lineage that led to ChatGPT.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): in its training data
- Claude Opus 5.5 (training cutoff June 2026): in its training data
- Gemini 4 Hydrogen (training cutoff March 2026): in its training data
- Grok 4.7 (training cutoff May 2026): in its training data

## Sources

1. [Improving language understanding with unsupervised learning (OpenAI)](https://openai.com/index/language-unsupervised/) (openai.com, official)
2. [Paper PDF](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) (cdn.openai.com, paper)

## Changes

- 2026-09-29 (filed): Created

## Related

- 2019-02-14: [OpenAI announces GPT-2 and withholds the full model over misuse concerns](https://postcutoff.com/e/2019-02-14-gpt-2/index.md)
- 2017-06-12: ['Attention Is All You Need' introduces the Transformer](https://postcutoff.com/e/2017-06-12-transformer/index.md)
- People: [Alec Radford](https://postcutoff.com/person/alec-radford/)