---
id: "2026-10-15-cammarata-goodfire-neural-networks-already-clean"
url: "https://postcutoff.com/e/2026-10-15-cammarata-goodfire-neural-networks-already-clean/"
as_of: "2026-10-16T02:16:00+02:00"
date: "2026-10-15"
date_precision: day
category: research
importance: 3
confidence: medium
status: [Partly confirmed]
sources: 9
editor: human editor
human_review: null
version: "2026-10-15"
---

As of: 2026-10-16 02:16 CEST.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2026-10-15-cammarata-goodfire-neural-networks-already-clean/

# Goodfire's Nick Cammarata argues 'Neural Networks Are Already Clean'

Full title: Goodfire's Nick Cammarata argues 'Neural Networks Are Already Clean': most neurons legible up to trillion-parameter models, superposition the wrong explanation

On Oct 15, 2026 Goodfire researcher Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI and started its 2023 automated interpretability project (~2% of GPT-2 XL neurons legible), published a 65-minute essay arguing that the field misread its canonical evidence for superposition. He claims most apparent polysemanticity comes from "recursive generalization" and fixable architecture and tooling problems, that roughly 90% of neuron poles in Qwen3-32B have one main job, and that similar results hold in models up to trillions of parameters such as Kimi K3. It is self-described as "an informal research essay and working hypothesis", not peer-reviewed or replicated.

## Key facts
- Self-description: "An informal research essay and working hypothesis about neural networks"; "A working hypothesis · 65-minute read"; a technical companion paper (with Curt Tigges) is announced as forthcoming
- Main claim: "The cause of polysemanticity in early model organisms wasn't superposition, but several ordinary, mostly fixable issues. We can now understand most neurons in models up to trillions of parameters"
- Cat/car neuron: InceptionV1 4e:55, the field's most famous polysemantic neuron, is reinterpreted as "not a cat and car neuron, it's a V-shape neuron" (inputs include fox snout, wolf face and van-front neurons; negative pole: round shapes)
- InceptionV1 last layer: ~5% interpretable with 2017-era tools (others said ~0%), ~50% with better tools on the same 2014 network, ~70% when retrained with modern regularization, "about 99%" in a ConvNeXt of the same size; versions trained "across a 300× range in size" stay "broadly legible at every scale"
- Qwen3-32B (preliminary audits): "roughly 90% of neuron poles have one main job, often with small related offshoots; another ~7% have two related jobs; and only ~2-4% are not yet understood"; the essay stresses the number depends on thresholds
- "We found similar results in models up to trillions of parameters, like Kimi K3, which the companion paper will cover in more detail"; it shows a neuron-only attribution graph of Kimi K3 (found-wallet prompt → "return") described as "the first published attribution graph of a multiple trillion parameter model" but adds "We haven't begun full audits of this scale yet"
- GPT-2 XL (model of OpenAI's 2023 automated interpretability project, then <2% clean): after re-running with 500 agent-written 'neuron papers', "about 70% look like roughly one thing"; "pristine" ones about 5%
- Methods: AI agents label neurons, build "explorable microscopes", write test inputs and review "100,000 attribution graphs"; a blind test on 183 Qwen3-32B neurons: agent-written examples from the explanation alone fired the neuron strongly 76% of the time vs 11% for near misses; 'most distant concept' texts fired ~1%
- Also reports a "double-negative regime" in SwiGLU neurons (gate and value both negative) that dominates the top activations of ~4% of neurons
- Stated limits: "We're maybe 85% sure"; the title "overstates the evidence, because we haven't audited everything"; frontier LLMs "aren't close to solved due to sheer size"; recursive generalization not shown to explain every polysemantic neuron
- Announcement X post (@nickcammarata): ~1.11M views, ~10.2k likes on Oct 15 (fxtwitter); he says he now thinks "ambitious interpretability will be solved by the time we need it most"
- Reactions on day one: Hacker News 374 points / 125 comments; LessWrong linkpost 121 karma; OpenAI's roon called it possibly "a major moment in scientific history" (~236k views); Eliezer Yudkowsky replied that no interpretability level makes aligning ASI "normal science" (~5–9k views per post)

## What happened

On Oct 15, 2026 Nick Cammarata of the interpretability company Goodfire published **"Neural Networks Are Already Clean"**, a long
interactive web essay (labelled "A working hypothesis · 65-minute read"). He co-authored the Distill circuits papers with Chris Olah and
started OpenAI's 2023 automated interpretability project. The essay argues that the field misread the evidence that made
**superposition** (networks folding unrelated concepts into the same neurons "simply because they don't interfere") the standard
explanation for polysemantic neurons. Mainstream interpretability therefore moved to decompositions such as sparse autoencoders.

Claims, in the essay's words:
- **The canonical examples were misread.** InceptionV1's famous cat-and-car neuron 4e:55 is "a V-shape neuron". The car-into-dogs circuit
  from *Zoom In* (2020), "the only one we showed, and the only one I ever saw", "wasn't hiding anything; it was ordinary generalization and
  composition."
- **Architecture and tools, not a second learning principle.** "Architectural problems made later layers overflow. Weak interpretability
  tools amplified these problems". With modern changes and "no change to model size or training data, 99-100% of neurons in that last layer
  are easily interpretable". Squeezed networks get coarser neurons, not arbitrary mixtures.
- **Recursive generalization.** Most remaining polysemanticity is "a motif I'll call recursive generalization": "as a neuron grows, it
  sprouts offshoots the way a tree branch sprouts sub-branches" (an "iy" neuron grows into "diy" and then "do it yourself"). He credits the
  "conjoined neurons" section of *Multimodal Neurons* (2021) with an earlier version of the idea.
- **Language models.** Preliminary audits of Qwen3-32B find "roughly 90% of neuron poles" with one main job. Qwen3.6-35B-A3B (a
  mixture-of-experts model) looks similar. "We found similar results in models up to trillions of parameters, like Kimi K3". The essay
  calls these numbers threshold-dependent and "a broad pointer". Re-running GPT-2 XL, the model of the 2023 OpenAI project, now gives
  "about 70%" of neurons that "look like roughly one thing".
- **Agents as the method.** "Agents can map out neurons and circuits, build explorable microscopes and explore them, test explanations
  thoroughly, and track hundreds of thousands of attribution graphs." He envisions "a laboratory run by automated scientists" guided by "a
  mechanistic constitution". Claude grouped the neurons in the essay's Qwen3-32B attribution graph.
- **Outlook.** "Highly speculative: this might be most of what we need, and interpretability might be far more tractable, possibly in the
  near term, than we've assumed." He calls this "a bitter lesson for interpretability: scaling ordinary investigation beats engineering a
  clever new decomposition."

Stated limits: the language-model detail is left to a "technical companion paper" written with Goodfire's Curt Tigges, which is not out
yet. Trillion-parameter models have not had "full audits". Frontier models "aren't close to solved due to sheer size". In "On the Title" the
authors say they are "maybe 85% sure" and that the title "overstates the evidence, because we haven't audited everything." The essay
thanks Anthropic's Joshua Batson and Jack Lindsey for feedback, and OpenAI's Jakub Pachocki among others for discussions, and quotes
Batson saying that the missing pieces "will fit into the initial puzzle, rather than being totally randomly splayed on top." No
independent replication exists yet.

Cammarata's X announcement (~1.11M views on day one) said he had been pessimistic after the early OpenAI work. It ends: "I now think
ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving."

## Why it matters

If the claims hold, the main approach to mechanistic interpretability would shift from training decompositions such as SAEs to reading a
network's own neurons with many AI agents. Interpretability would also be more tractable for frontier-scale models than most researchers
assumed. The essay comes from a founder of the circuits research programme and revises its own canonical examples. It has not been peer
reviewed: the quantitative claims are preliminary audits by the authors, and the companion paper is not yet public.

The essay stands in contrast to other interpretability researchers' recent public statements. In Palisade Research's frominside.ai
interview (released 2026-09-29), Google DeepMind's interpretability lead Neel Nanda said: "Speaking as an interpretability expert, please
do not rely on us to save you on the current trajectory." The disagreement over whether better interpretability makes alignment "normal
science" (roon versus Yudkowsky) is separate from the empirical question of how legible neurons are.

## Reactions

First day.

- **roon** (OpenAI) quoted the post: "this may be remembered as a major moment in scientific history. news so good even eliezer will
  have to lower his pdoom. we can start making alignment into a 'normal science'" (~236k views).
- **Eliezer Yudkowsky** replied that "If you think there is any possible level of interpretability that makes 'align ASI' be 'normal
  science' … then you do not understand what ASI is". He called the essay's examples "utterly. fucking. primitive. playgrounds" relative to
  "understanding how a mind is thinking" (three posts, ~4.9k–8.7k views each).
- **Goodfire colleagues** Curt Tigges ("neurons are far more interpretable than we thought"; he adds that superposition is not
  "impossible" but "probably isn't optimal most of the time") and chief scientist Tom McGrath ("we're on the cusp of huge breakthroughs in
  interpretability") posted support; both posts had small reach (under 2k views).
- **Hacker News** (374 points, 125 comments): mixed. Some called it a possible "watershed moment". Others wrote that "the claims seem to be
  an overreach of what is actually proven" and that "there are good theoretical reasons to think superposition exists … particularly in
  the residual stream". One commenter compared the tone to the "hypomanic energy" around Anthropic's first large sparse-autoencoder results.
  Many complained about the length and the heavy interactive page.
- **LessWrong** linkpost (121 karma): Thane Ruthenis pre-registered "default skepticism", arguing that the main problem is not knowing how
  trained networks turn inputs into abstract concepts. Others discussed whether easier interpretability would also speed up capabilities.
- As of Oct 15 no public response from Chris Olah, Neel Nanda or Anthropic's interpretability team was found. Search covered fxtwitter
  timelines of @ch402 and @NeelNanda5, plus HN and LessWrong.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): 168 days after its cutoff
- Claude Opus 5.5 (training cutoff June 2026): 107 days after its cutoff
- Gemini 4 Argon (training cutoff March 2026): 198 days after its cutoff
- Grok 4.7 (training cutoff May 2026): 137 days after its cutoff

## Sources

1. [Nick Cammarata (Goodfire): Neural Networks Are Already Clean](https://already-clean.goodfire.com/) (already-clean.goodfire.com, official)
2. [Background: Bills, Cammarata et al. (OpenAI, 2023), Language models can explain neurons in language models](https://openai.com/research/language-models-can-explain-neurons-in-language-models) (openai.com, paper)
3. [Background: Olah, Cammarata et al. (Distill, 2020), Zoom In: An Introduction to Circuits](https://distill.pub/2020/circuits/zoom-in/) (distill.pub, paper)
4. [Nick Cammarata's announcement on X (~1.1M views)](https://x.com/nickcammarata/status/2110757819427574056) (x.com, discussion)
5. [Hacker News discussion (374 points)](https://news.ycombinator.com/item?id=50105647) (news.ycombinator.com, discussion)
6. [LessWrong linkpost and comments](https://www.lesswrong.com/posts/x4kvabkF2Jk8GLZp8/linkpost-neural-networks-are-already-clean) (lesswrong.com, discussion)
7. [roon (OpenAI): 'may be remembered as a major moment in scientific history'](https://x.com/tszzl/status/2110758311339729061) (x.com, discussion)
8. [Eliezer Yudkowsky's reply to roon](https://x.com/allTheYud/status/2110804218005332178) (x.com, discussion)
9. [Curt Tigges (Goodfire, co-worker on the language-model part) thread](https://x.com/CurtTigges/status/2110806833724330256) (x.com, discussion)

## Changes

- 2026-10-15 (filed): Created (essay read in full from the page text; reactions from HN Algolia, LessWrong GraphQL and api.fxtwitter.com)

## Related

- 2026-10-08: [Goodfire launches 'inside-out' activation-probe monitors for AI agents](https://postcutoff.com/e/2026-10-08-goodfire-activation-probe-monitors/index.md)
- 2026-07-16: [Moonshot AI releases Kimi K3, a 2.8T-parameter open-weights multimodal model](https://postcutoff.com/e/2026-07-16-moonshot-kimi-k3/index.md)
- 2026-07-06: [Anthropic finds a "global workspace" inside Claude using a Jacobian lens](https://postcutoff.com/e/2026-07-06-anthropic-global-workspace-j-lens/index.md)
- 2026-05-07: [Anthropic introduces Natural Language Autoencoders that translate model activations into readable text](https://postcutoff.com/e/2026-05-07-anthropic-natural-language-autoencoders/index.md)
- 2026-04-02: [Anthropic finds functional emotion representations that causally drive Claude's behavior](https://postcutoff.com/e/2026-04-02-anthropic-emotion-concepts-interpretability/index.md)
- People: [Chris Olah](https://postcutoff.com/person/chris-olah/), [Eliezer Yudkowsky](https://postcutoff.com/person/eliezer-yudkowsky/), [Jakub Pachocki](https://postcutoff.com/person/jakub-pachocki/), [Neel Nanda](https://postcutoff.com/person/neel-nanda/), [Nick Cammarata](https://postcutoff.com/person/nick-cammarata/), [roon](https://postcutoff.com/person/roon/)
- Archived: 3 posts, listed in https://postcutoff.com/e/2026-10-15-cammarata-goodfire-neural-networks-already-clean/index.json