---
id: "2026-11-02-gpt-6-astra-edits-qwen-weights-weight-space-programming"
url: "https://postcutoff.com/e/2026-11-02-gpt-6-astra-edits-qwen-weights-weight-space-programming/"
as_of: "2026-11-09T21:13:00+01:00"
date: "2026-11-02"
date_precision: day
category: research
importance: 4
confidence: high
status: [Confirmed]
sources: 6
editor: human editor
human_review: null
version: "2026-11-04"
---

As of: 2026-11-09 21:13 CET.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2026-11-02-gpt-6-astra-edits-qwen-weights-weight-space-programming/

# MIT CSAIL researchers

Full title: MIT CSAIL researchers: GPT-6 Astra read and hand-edited Qwen3.5-0.8B's weights, lifting MATH-500 from 7.9% to 48.1% with no training

On Nov 2, 2026 Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL) published 'weight-space programming': GPT-6 Astra, working in a Python session with 50 forward passes and no gradient descent, found that Qwen3.5-0.8B's failures came largely from looping and edited 131,072 weights (0.016%) in two MLP layers. MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. The post reached 2,758 points on Hacker News.

## Key facts
- Edit: Astra traced looping to one direction written by the MLPs in layers 15 and 19 and subtracted it from 64 output columns in each layer: 131,072 weights, 0.016% of the model
- Base → edited (50 forward passes): MATH-500 7.9% → 48.1%; GSM8K 13.8% → 67.1%; IFEval 21.0% → 62.3%; answers hitting the 32,768-token limit 93% → 33.1%
- With 10 or 25 forward passes the gains were much smaller
- Limits stated by the authors: 'very early results'; Astra could not improve Gemma 3 270M (which rarely loops); it could not identify a sycophantic model from weights alone; some experiments use small samples; Qwen's looping problem is documented in its model card
- Reach: 2,758 points on Hacker News (Nov 2), Techmeme lead story on Nov 3
- Lead author's X announcement: ~6.53M views (fxtwitter, Nov 3)

## What happened

The authors gave GPT-6 Astra raw weights of small open models and asked it to change their behaviour by writing new weight values directly, allowed to run forward passes to test ideas but not to train. The headline result is the Qwen3.5-0.8B maths edit; other experiments (sycophancy detection, a diffusion model) were mixed, and the authors say the edit removes an obstacle rather than adding an ability the model lacked. The result is a blog post, not a peer-reviewed paper.

## Why it matters

A frontier model directly reading and rewriting another network's parameters is a concrete, measurable step toward models modifying models, and the sharp reaction shows how closely this is watched in the RSI debate.

## Criticism (Nov 2–3)

Matthew Honnibal (spaCy; 996 HN points) argues the post conflates techniques: only three experiments read raw weights with no tools, on
networks of 17K–172K weights, while the Qwen result relied on Python code and running the model on practice batches. "It's not a single
technique," he writes. He notes the MinAtar win holds only on Breakout (517 vs 25; losses on Seaquest 53 vs 89 and Asterix 53 vs 55), and
that the Gemma 3 sycophancy detection and the transfer of the Qwen fix to Gemma 3 failed. Simon Willison (650 HN points) showed a local
Qwen 3.8 27B doing similar small-network interpretation, and Sean Goedecke argued the abilities are not superhuman.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): 186 days after its cutoff
- Claude Opus 5.5 (training cutoff June 2026): 125 days after its cutoff
- Gemini 4 Argon (training cutoff March 2026): 216 days after its cutoff
- Grok 4.7 (training cutoff May 2026): 155 days after its cutoff

## Sources

1. [Takehi, Teoh, Zhang, Khattab: Weight-space programming (blog, Nov 2)](https://rikiyat.github.io/blog/weight-space-programming/) (rikiyat.github.io, paper)
2. [Matthew Honnibal: The 'Astra can read and write weights' post is misleading (Nov 3)](https://honnibal.dev/blog/astra-read-write-misleading) (honnibal.dev, discussion)
3. [Simon Willison: Qwen 3.8 27B on my laptop can interpret model weights, too (Nov 3)](https://simonwillison.net/2026/Nov/3/qwen-can-interpret-model-weights/) (simonwillison.net, discussion)
4. [Sean Goedecke: LLMs are not superhuman at reading and writing neural network weights](https://www.seangoedecke.com/llms-are-not-superhuman-at-reading-and-writing-neural-network-weights/) (seangoedecke.com, discussion)
5. [Hacker News discussion (2,758 points)](https://news.ycombinator.com/item?id=50341174) (news.ycombinator.com, discussion)
6. [@rikiyatakehi on X (Nov 2)](https://x.com/rikiyatakehi/status/2117313742891327994) (x.com, discussion)

## Changes

- 2026-11-04: Added Honnibal, Willison and Goedecke critiques
- 2026-11-03: X reach added
- 2026-11-03 (filed): Created

## Related

- 2026-10-14: [Gladstone report: insiders expect RSI in 3–12 months](https://postcutoff.com/e/2026-10-14-gladstone-endgame-what-rsi-looks-like-inside-labs/index.md)
- People: [Simon Willison](https://postcutoff.com/person/simon-willison/)
- Archived: 1 post, listed in https://postcutoff.com/e/2026-11-02-gpt-6-astra-edits-qwen-weights-weight-space-programming/index.json