--- id: "2026-09-21-dettmers-dlab-open-source-week" url: "https://postcutoff.com/e/2026-09-21-dettmers-dlab-open-source-week/" as_of: "2026-11-19T22:11:00+01:00" date: "2026-09-21" date_precision: day category: open-source importance: 2 confidence: medium status: [Partly confirmed] sources: 5 editor: human editor human_review: null version: "2026-10-07" --- As of: 2026-11-19 22:11 CET. Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/ Canonical page: https://postcutoff.com/e/2026-09-21-dettmers-dlab-open-source-week/ # Tim Dettmers' dlab open-source week: 1.5-bit inference and a 125B model on one 24 GB GPU Full title: Tim Dettmers' dlab open-source week: 1.5-bit inference, a 125B model on one 24 GB GPU, CliffCompaction and a local research agent On Sept 21, 2026 quantization researcher Tim Dettmers announced "dlab Open Source Week: Frontier AI on Your Own Hardware": two open-source projects and four papers released over the following days. Headline claims: a 125B Qwen 3.8 Flash Next running on a single 24 GB GPU, a Qwen 3.6 35B-A3B at 1.5 bits per weight reaching ~450 tokens/s on Apple Metal, and a private beta of bitsandbytes2. There is also CliffCompaction, an auto-compaction proxy for long coding-agent runs (arXiv 2609.26779), and a local autonomous-research agent he says beats frontier labs' deep-research systems. ## Key facts - Announcement post dated Sept 21, 2026 on timdettmers.com; releases followed from Sept 22 - Inference framework: Qwen 3.8 Flash Next (125B) on a single 24 GB GPU; DeepSeek V4.1 (~550B) on an AMD Strix, DGX Spark or 128 GB MacBook (author's claims) - Compression: Qwen 3.6 35B-A3B at 1.5 bits/weight, ~450 tokens/s, about a tenth of the memory of 16-bit weights; bitsandbytes2 (bnb2) opened as a private beta with 'runtime dynamic compression' - CliffCompaction (Nguyen, Cho, Chen, Dettmers; arXiv 2609.26779, Sept 22): up to ~50% lower cost under a bounded context with maintained or better Terminal-Bench results; KernelBench CUDA speedups 2.23× after 200 steps and 3.58× after 400; open-source API proxy works with Claude Code and Codex - Agent sessions reportedly run past 100M tokens; one company reported a 45% cut in its AI budget - Unverified claim: the autonomous research system 'beats deep research systems from frontier labs' and beats Sakana AI's system and Google's ScientistOne. No independent evaluation found yet ## What happened Tim Dettmers, author of QLoRA and the bitsandbytes library, used a week of releases from his lab to argue that near-frontier AI can run on hardware people own. The pieces were an inference framework (with a Mac/Metal path), aggressive low-bit compression through bitsandbytes2, an agent harness, CliffCompaction for very long agent sessions, and an autonomous research agent. The post drew attention on Hacker News (185 points). ## Why it matters If the claims hold, 1.5-bit weights and single-consumer-GPU inference of 100B+ MoE models push open-weights AI further out of datacenters. That matters both for access and for the debate about controlling open models. The comparison with frontier deep-research systems comes from the author and should be treated as unverified. ## Your AI and this story - GPT-6.2 Astra (training cutoff May 2026): 113 days after its cutoff - Claude Opus 5.5 (training cutoff June 2026): 83 days after its cutoff - Gemini 4 Hydrogen (training cutoff March 2026): 174 days after its cutoff - Grok 4.7 (training cutoff May 2026): 113 days after its cutoff ## Sources 1. [Tim Dettmers: dlab Open Source Week, Frontier AI on Your Own Hardware](https://timdettmers.com/2026/09/21/dlab-open-source-week/) (timdettmers.com, official) 2. [arXiv 2609.26779: CliffCompaction, cost-efficient compaction for long-horizon coding agents](https://arxiv.org/abs/2609.26779v1) (arxiv.org, paper) 3. [GitHub: cliffcompaction](https://github.com/nguyenvuthientrang/cliffcompaction) (github.com, code) 4. [LAVX News: dlab Open Source Week brings frontier AI to ordinary hardware](https://news.lavx.hu/article/dlab-open-source-week-brings-frontier-ai-to-ordinary-hardware) (news.lavx.hu, press) 5. [Tim Dettmers on X: first release, runtime dynamic compression and bitsandbytes2 private beta](https://x.com/Tim_Dettmers/status/2102418118568018281) (x.com, discussion) ## Changes - 2026-09-30 (filed): Created (only the CliffCompaction paper was checked on arXiv, the other three papers were not located) ## Related - People: [Tim Dettmers](https://postcutoff.com/person/tim-dettmers/) - Archived: 1 post, listed in https://postcutoff.com/e/2026-09-21-dettmers-dlab-open-source-week/index.json