Meta FAIR and collaborators propose 'Context Language Models' that edit their own context as a file
On Sept 29, 2026 a paper titled "Context Language Models" (arXiv 2609.37725, code at facebookresearch/context-language-models) proposed language models that manage their own context natively. The model treats its context as a file and can rewrite it freely, so the agent harness no longer decides what to keep. The authors report 11.4% higher BrowseComp-Plus accuracy with 21.5% fewer FLOPs, and a 47.6% BrowseComp-Plus gain for Qwen3.5-9B after online RL, while using 12% fewer FLOPs.
Key facts
- arXiv 2609.37725, submitted Sept 29, 2026. Authors include Rulin Shao, Hamish Ivison, Nathan Lambert, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer and Pang Wei Koh
- Idea: the context is an editable file that the model updates without restriction. This allows both in-context and parametric (RL-trained) learning of context-management strategies, and supports multi-agent setups
- Reported results: BrowseComp-Plus +11.4% accuracy with 21.5% fewer FLOPs; 12-hour EdgeBench +5% with 59% fewer FLOPs; multi-repository agent task, 65% greater improvement at the same compute
- Online RL on Qwen3.5-9B: +47.6% on BrowseComp-Plus with 12% fewer FLOPs
- Official code: github.com/facebookresearch/context-language-models
What happened
A team centred on Meta FAIR, with authors who also list academic and Ai2 affiliations, released "Context Language Models" (CLMs). In this design the model does not rely on an external harness to summarise or truncate its context. It edits the context directly, the way an agent edits a file, and RL can teach it good context-maintenance strategies. The Neuron newsletter (Sept 30) highlighted the work.
Why it matters
Context management (compaction, memory files, sub-agents) became a central engineering problem for long-running agents in 2026. CLMs move that job into the model itself, and the reported compute savings come on deep-research and long-horizon agent benchmarks.
Unverified: the exact affiliation of each author was not checked (org list is approximate). The results are the authors' own and have not been peer-reviewed.
Changelog
- 2026-10-01: created (leads run, 07:40 completion)
Sources (2)
id: 2026-09-29-meta-fair-context-language-models · updated 2026-10-01 · open in the interactive timeline