---
id: "2026-10-07-bolmo-byteification-nature"
url: "https://postcutoff.com/e/2026-10-07-bolmo-byteification-nature/"
as_of: "2026-10-12T07:22:00+02:00"
date: "2026-10-07"
date_precision: day
category: research
importance: 2
confidence: high
status: [Confirmed]
sources: 5
editor: Adam Bicz
human_review: null
version: "2026-10-11"
---

As of: 2026-10-12 07:22 CEST.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code). Human editor: Adam Bicz.
Canonical page: https://postcutoff.com/e/2026-10-07-bolmo-byteification-nature/

# Nature publishes Ai2's 'byteification' paper

Full title: Nature publishes Ai2's 'byteification' paper: Bolmo retrofits existing subword LLMs into byte-level models

On 7 Oct 2026 Nature published "Retrofitting language models to operate over bytes" by Benjamin Minixhofer, Luca Soldaini, Valentin Hofmann, Luke Zettlemoyer, Noah A. Smith, Anna Korhonen, Edoardo M. Ponti, Tyler Murray and Tomasz Limisiewicz (Ai2-led, with university co-authors). The method, byteification, converts an existing subword LLM into a byte-level one with a two-stage procedure and little extra training. The resulting fully open Bolmo models (released by Ai2 in December 2025, arXiv 2512.15586) beat earlier byte-level models and excel on character-level tasks. Nature ran a News & Views piece alongside it.

## Key facts
- Nature paper: doi 10.1038/s41586-026-11111-4, published online 7 Oct 2026; Nature's summary: byteification 'is a general method for retrofitting large language models to operate over bytes such that they approach the capabilities of subword-based systems and may be better for scientific data'
- Preprint: 'Bolmo: Byteifying the Next Generation of Language Models', arXiv 2512.15586 (v1 17 Dec 2025, v2 9 Feb 2026), CC BY 4.0
- Abstract: subword tokenization 'obscures fine-grained information, which is problematic, especially for scientific data - such as computer code or biological sequences'; the byteified models 'outperform prior byte-level approaches and excel on character-level reasoning tasks, while remaining competitive across standard benchmarks', with 'practical inference speeds'
- Weights: allenai/Bolmo-7B on Hugging Face (Apache-2.0; base model Olmo 3 7B)
- News & Views (Zhao Zhang and Yingfei Xiong, Nature, 7 Oct): 'Retrofitted LLM can count the letter i's in artificial intelligence'
- Secondary reports (AI Weekly digests, not checked against the paper): conversion uses under 1% of a pretraining run (49.1B tokens); byteified variants also of Qwen 3 8B and Llama 3 8B; Bolmo 7B +16.5 points on STEM over BLT 7B

## What happened

Most LLMs read text as subword tokens, which hides individual letters; this is why models famously miscount letters in words and
struggle with some code and biological sequences. Byte-level models avoid this but had trailed tokenized models. The Ai2-led team
showed that an existing subword model can be "byteified": a two-stage conversion keeps most of what the model learned and makes it
read raw bytes. Ai2 released the Bolmo models in December 2025; the peer-reviewed version appeared in Nature on 7 Oct 2026.

## Why it matters

It is a peer-reviewed, fully open route to tokenizer-free models that reuses existing checkpoints instead of training from scratch,
with possible benefits for scientific data where single characters matter.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): 160 days after its cutoff
- Claude Opus 5.5 (training cutoff June 2026): 99 days after its cutoff
- Gemini 3.8 Flash (training cutoff March 2026): 190 days after its cutoff
- Grok 4.7 (training cutoff May 2026): 129 days after its cutoff

## Sources

1. [Nature: Retrofitting language models to operate over bytes (Minixhofer et al.)](https://www.nature.com/articles/s41586-026-11111-4) (nature.com, paper)
2. [arXiv 2512.15586: Bolmo: Byteifying the Next Generation of Language Models](https://arxiv.org/abs/2512.15586) (arxiv.org, paper)
3. [Hugging Face: allenai/Bolmo-7B](https://huggingface.co/allenai/Bolmo-7B) (huggingface.co, code)
4. [Nature News & Views: Retrofitted LLM can count the letter 'i's in 'artificial intelligence'](https://www.nature.com/articles/d41586-026-03059-2) (nature.com, press)
5. [AI Weekly: Bolmo byteifies OLMo, Llama and Qwen at under 1% pretrain cost](https://aiweekly.co/alerts/bolmo-byteifies-olmo-llama-and-qwen-at-under-1-pretrain-cost) (aiweekly.co, press)

## Changes

- 2026-10-11 (filed): Created