Qwen3.8-Flash-Next, a 125B MoE with 6B active, previews the Qwen 4 architecture
Partly confirmed
Status
- Claim
Partly confirmed
- Our reporting
- Medium confidence
- Importance
- 3 of 5
- Last verified
- 4 October 2026
Your AI and this story
- GPT-6 Astra118 days after its cutoff
- Claude Opus 5.557 days after its cutoff
- Gemini 4 Hydrogen148 days after its cutoff
- Grok 4.787 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 57 days before it.
Key facts
- 125B backbone + 51B n-gram embeddings + 4B multi-token-prediction = ~180B on disk; 6B active per token
- 512 experts, 10 routed + 1 shared per token; Gated DeltaNet in 3 of 4 layers + Qwen Sparse Attention
- Context: 262,144 native, 1M with YaRN
- Reported benchmarks: SWE-bench Pro 62.5, AndroidWorld 84.5, MathVision 95.7
- Training cost ~1/9 of Qwen3.7-Plus; up to 7.6x prefill and 4.9x decode speedup at 1M tokens
- License: qwen-community-1.0 (not Apache 2.0)
- Local inference: the community project Strata (github.com/Niko1221/Strata, MIT, created Sept 24, 2026; ~10.5k stars by Oct 4) packages 2–3-bit GGUF quants with a one-click installer for 12 GB+ NVIDIA/AMD cards and 32–64 GB RAM; its README reports 94 tok/s output on an RTX 5070 (Q2_0); it reached 430 points on Hacker News on Oct 4
- Strata’s claims are self-reported: an HN commenter measured much worse vision accuracy than llama.cpp with the same weights, and others questioned quality below 4-bit; the submitter reported 124 tok/s on an RTX 4090 with 128 GB RAM
What happened
Weights for Qwen3.8-Flash-Next landed on Hugging Face and ModelScope (BF16 and FP8) on 2026-08-26. The model combines an extreme sparsity ratio (6B of 125B active), a 20M-entry n-gram embedding table, and linear-attention (Gated DeltaNet) layers interleaved with sparse attention — the Qwen team presented it as an early look at Qwen 4 so developers can prepare tooling.
Why it matters
It pushes the cost frontier: near-frontier agentic coding numbers at 6B active parameters make strong models cheap to serve at 1M-token contexts.
Sources
8 sources from 7 sites. Numbers match the chips in the text.
8 sources: 4 primary, 3 press, 1 reaction
Primary
- Qwen blog - Qwen3.8-Flash-Next: A New Architecture, Towards Ultimate Cost-Efficiencyqwen.ai, official
- Qwen3.8-Flash-Next tech report (GitHub)github.com, paper
- Hugging Face - Qwen/Qwen3.8-Flash-Nexthuggingface.co, code
- GitHub: Strata, Qwen3.8-Flash-Next on consumer hardware (community)github.com, code
Press
- Bloomberg: Alibaba releases smaller, cost-effective Qwen AI modelbloomberg.com, press
- MarkTechPost: Qwen3.8-Flash-Next technical breakdownmarktechpost.com, press
- The Decoder: Qwen3.8-Flash-Next targets ultimate cost efficiencythe-decoder.com, press
Reactions
- Hacker News: Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s (430 pts)news.ycombinator.com, discussion
Changes
- Added the community Strata local-inference project (HN front page, 430 pts) and caveats from the HN thread
- Added the official Qwen blog post, tech report and Hugging Face links
- Filed