Nature publishes Ai2’s ‘byteification’ paper
Bolmo retrofits existing subword LLMs into byte-level models
Confirmed
The takeaway
Byte-level language models, which read raw bytes instead of tokens, had lagged behind tokenized models. A Nature paper shows existing models can be converted cheaply and keep most of their ability while gaining character-level skills.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 2 of 5
- Last verified
- 11 October 2026
Your AI and this story
- GPT-6 Astra160 days after its cutoff
- Claude Opus 5.599 days after its cutoff
- Gemini 3.8 Flash190 days after its cutoff
- Grok 4.7129 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 99 days before it.
Key facts
- Nature paper: doi 10.1038/s41586-026-11111-4, published online 7 Oct 2026; Nature’s summary: byteification ‘is a general method for retrofitting large language models to operate over bytes such that they approach the capabilities of subword-based systems and may be better for scientific data’
- Preprint: ‘Bolmo: Byteifying the Next Generation of Language Models’, arXiv 2512.15586 (v1 17 Dec 2025, v2 9 Feb 2026), CC BY 4.0
- Abstract: subword tokenization ‘obscures fine-grained information, which is problematic, especially for scientific data - such as computer code or biological sequences’; the byteified models ‘outperform prior byte-level approaches and excel on character-level reasoning tasks, while remaining competitive across standard benchmarks’, with ‘practical inference speeds’
- Weights: allenai/Bolmo-7B on Hugging Face (Apache-2.0; base model Olmo 3 7B)
- News & Views (Zhao Zhang and Yingfei Xiong, Nature, 7 Oct): ‘Retrofitted LLM can count the letter i’s in artificial intelligence’
- Secondary reports (AI Weekly digests, not checked against the paper): conversion uses under 1% of a pretraining run (49.1B tokens); byteified variants also of Qwen 3 8B and Llama 3 8B; Bolmo 7B +16.5 points on STEM over BLT 7B
What happened
Most LLMs read text as subword tokens, which hides individual letters; this is why models famously miscount letters in words and struggle with some code and biological sequences. Byte-level models avoid this but had trailed tokenized models. The Ai2-led team showed that an existing subword model can be “byteified”: a two-stage conversion keeps most of what the model learned and makes it read raw bytes. Ai2 released the Bolmo models in December 2025; the peer-reviewed version appeared in Nature on 7 Oct 2026.
Why it matters
It is a peer-reviewed, fully open route to tokenizer-free models that reuses existing checkpoints instead of training from scratch, with possible benefits for scientific data where single characters matter.
Sources
5 sources from 4 sites. Numbers match the chips in the text.
5 sources: 3 primary, 2 press
Primary
- Nature: Retrofitting language models to operate over bytes (Minixhofer et al.)nature.com, paper
- arXiv 2512.15586: Bolmo: Byteifying the Next Generation of Language Modelsarxiv.org, paper
- Hugging Face: allenai/Bolmo-7Bhuggingface.co, code
Press
Changes
- Filed