{"schema":"postcutoff/event@1","as_of":"2026-11-19T22:11:00+01:00","url":"https://postcutoff.com/e/2026-09-21-dettmers-dlab-open-source-week/","md":"https://postcutoff.com/e/2026-09-21-dettmers-dlab-open-source-week/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-09-21-dettmers-dlab-open-source-week","date":"2026-09-21","date_precision":"day","short_title":"Tim Dettmers' dlab open-source week: 1.5-bit inference and a 125B model on one 24 GB GPU","deck":null,"takeaway":"On Sept 21, 2026 quantization researcher Tim Dettmers announced \"dlab Open Source Week: Frontier AI on Your Own Hardware\": two open-source projects and four papers released over the following days.","category":"open-source","category_label":"Open source","importance":2,"confidence":"medium","status":{"key":"partly","labels":["Partly confirmed"]},"sources":[{"n":1,"title":"Tim Dettmers: dlab Open Source Week, Frontier AI on Your Own Hardware","url":"https://timdettmers.com/2026/09/21/dlab-open-source-week/","type":"official","group":"primary","domain":"timdettmers.com"},{"n":2,"title":"arXiv 2609.26779: CliffCompaction, cost-efficient compaction for long-horizon coding agents","url":"https://arxiv.org/abs/2609.26779v1","type":"paper","group":"primary","domain":"arxiv.org"},{"n":3,"title":"GitHub: cliffcompaction","url":"https://github.com/nguyenvuthientrang/cliffcompaction","type":"code","group":"primary","domain":"github.com"},{"n":4,"title":"LAVX News: dlab Open Source Week brings frontier AI to ordinary hardware","url":"https://news.lavx.hu/article/dlab-open-source-week-brings-frontier-ai-to-ordinary-hardware","type":"press","group":"press","domain":"news.lavx.hu"},{"n":5,"title":"Tim Dettmers on X: first release, runtime dynamic compression and bitsandbytes2 private beta","url":"https://x.com/Tim_Dettmers/status/2102418118568018281","type":"discussion","group":"reactions","domain":"x.com"}],"official":3,"filed":"2026-09-30","updated":"2026-09-30","orgs":["dlab","Carnegie Mellon University"],"title":"Tim Dettmers' dlab open-source week: 1.5-bit inference, a 125B model on one 24 GB GPU, CliffCompaction and a local research agent","summary":"On Sept 21, 2026 quantization researcher Tim Dettmers announced \"dlab Open Source Week: Frontier AI on Your Own Hardware\": two open-source projects and four papers released over the following days. Headline claims: a 125B Qwen 3.8 Flash Next running on a single 24 GB GPU, a Qwen 3.6 35B-A3B at 1.5 bits per weight reaching ~450 tokens/s on Apple Metal, and a private beta of bitsandbytes2. There is also CliffCompaction, an auto-compaction proxy for long coding-agent runs (arXiv 2609.26779), and a local autonomous-research agent he says beats frontier labs' deep-research systems.","key_facts":["Announcement post dated Sept 21, 2026 on timdettmers.com; releases followed from Sept 22","Inference framework: Qwen 3.8 Flash Next (125B) on a single 24 GB GPU; DeepSeek V4.1 (~550B) on an AMD Strix, DGX Spark or 128 GB MacBook (author's claims)","Compression: Qwen 3.6 35B-A3B at 1.5 bits/weight, ~450 tokens/s, about a tenth of the memory of 16-bit weights; bitsandbytes2 (bnb2) opened as a private beta with 'runtime dynamic compression'","CliffCompaction (Nguyen, Cho, Chen, Dettmers; arXiv 2609.26779, Sept 22): up to ~50% lower cost under a bounded context with maintained or better Terminal-Bench results; KernelBench CUDA speedups 2.23× after 200 steps and 3.58× after 400; open-source API proxy works with Claude Code and Codex","Agent sessions reportedly run past 100M tokens; one company reported a 45% cut in its AI budget","Unverified claim: the autonomous research system 'beats deep research systems from frontier labs' and beats Sakana AI's system and Google's ScientistOne. No independent evaluation found yet"],"key_numbers":[],"tags":["open-source","quantization","bitsandbytes","local-inference","agents","context-compaction","autonomous-research","consumer-hardware"],"science":null,"body_md":"## What happened\n\nTim Dettmers, author of QLoRA and the bitsandbytes library, used a week of releases from his lab to argue that near-frontier AI can\nrun on hardware people own. The pieces were an inference framework (with a Mac/Metal path), aggressive low-bit compression through\nbitsandbytes2, an agent harness, CliffCompaction for very long agent sessions, and an autonomous research agent. The post drew\nattention on Hacker News (185 points).\n\n## Why it matters\n\nIf the claims hold, 1.5-bit weights and single-consumer-GPU inference of 100B+ MoE models push open-weights AI further out of\ndatacenters. That matters both for access and for the debate about controlling open models. The comparison with frontier deep-research\nsystems comes from the author and should be treated as unverified.","disputed":[],"related":[],"people":[{"id":"tim-dettmers","name":"Tim Dettmers","url":"https://postcutoff.com/person/tim-dettmers/"}],"posts":[{"id":"x-tim_dettmers-2102418118568018281","title":"“This is the first release in our open-source week: runtime dynamic compression and the…”","url":"https://postcutoff.com/p/x-tim_dettmers-2102418118568018281/"}],"videos":[],"models":[],"changes":[{"date":"2026-09-30","type":"filed","text":"Created (only the CliffCompaction paper was checked on arXiv, the other three papers were not located)"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-09-30","run":null,"sources_read":null,"updated":"2026-09-30","human_review":null,"version":{"date":"2026-10-07"}},"gaps":[{"model_id":"gpt-6-2-astra","name":"GPT-6.2 Astra","cutoff":"2026-05","days_after":113,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":83,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":174,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":113,"in_training_data":false}],"short_url":null}