---
id: "2018-10-11-bert"
url: "https://postcutoff.com/e/2018-10-11-bert/"
as_of: "2026-11-13T13:49:00+01:00"
date: "2018-10-11"
date_precision: day
category: model-release
importance: 4
confidence: high
status: [Confirmed]
sources: 2
editor: human editor
human_review: null
version: "2026-10-04"
---

As of: 2026-11-13 13:49 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2018-10-11-bert/

# Google releases BERT, bidirectional Transformer pre-training

BERT pre-trained a bidirectional Transformer encoder with masked language modeling and set new records on 11 NLP tasks; it was open-sourced and soon deployed in Google Search.

## Key facts
- arXiv 1810.04805 (October 2018); NAACL 2019 best paper
- BERT-Large: 340M parameters
- Pre-training objectives: masked LM + next sentence prediction
- Google said in October 2019 that BERT was used in Search ranking

## What happened

Google published and open-sourced BERT, which fine-tuned easily to classification, QA and tagging tasks.

## Why it matters

Triggered the 'ImageNet moment' of NLP: pre-trained Transformers became the default for all language tasks.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): in its training data
- Claude Opus 5.5 (training cutoff June 2026): in its training data
- Gemini 4 Hydrogen (training cutoff March 2026): in its training data
- Grok 4.7 (training cutoff May 2026): in its training data

## Sources

1. [BERT: Pre-training of Deep Bidirectional Transformers (arXiv)](https://arxiv.org/abs/1810.04805) (arxiv.org, paper)
2. [google-research/bert (code)](https://github.com/google-research/bert) (github.com, code)

## Changes

- 2026-09-29 (filed): Created

## Related

- 2018-06-11: [OpenAI's GPT-1: generative pre-training of Transformers](https://postcutoff.com/e/2018-06-11-gpt-1/index.md)
- 2017-06-12: ['Attention Is All You Need' introduces the Transformer](https://postcutoff.com/e/2017-06-12-transformer/index.md)