---
id: "2026-10-16-nvidia-nemotron-labs-dory-looped-4b"
url: "https://postcutoff.com/e/2026-10-16-nvidia-nemotron-labs-dory-looped-4b/"
as_of: "2026-10-17T07:29:00+02:00"
date: "2026-10-16"
date_precision: day
category: open-source
importance: 3
confidence: high
status: [Confirmed]
sources: 3
editor: human editor
human_review: null
version: null
---

As of: 2026-10-17 07:29 CEST.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/e/2026-10-16-nvidia-nemotron-labs-dory-looped-4b/

# Nvidia Dory 1.0: open 4B looped language model

Full title: Nvidia releases Nemotron Labs Dory 1.0, an open 4B looped (latent-recurrent) LM whose loop count trades compute for accuracy

On Oct 16, 2026 Nvidia published NVIDIA-Nemotron-Labs-Dory1.0-4B on Hugging Face (OpenMDW-1.1 license): a 4B looped language model that applies a shared 20-layer recurrent block several times per token, so one checkpoint can run with 1, 2 or 4 loops and spend more compute "thinking in latent space" instead of writing longer outputs. With 4 loops, Nvidia's card shows SWE-bench Verified rising from 50.3 to 61.8 and Terminal-Bench 2.1 from 12.5 to 22.2. It is pre-trained natively in NVFP4 and has a last-loop KV cache, so looping costs no extra cache memory.

## Key facts
- Architecture (model card): looped normalized-GPT (nGPT) with interleaved sliding-window and full attention; layer stack = Input block (8 layers), shared Recurrent block (20 layers) applied repeatedly with the input reinjected each loop, Output block (8 layers); 4B parameters; up to 256K context; English
- Elastic inference: the number of loops is set at inference time, with no retraining or model swap
- Last-loop KV caching (recurrent_kv_cache_mode = 'last_loop'): one cache per recurrent attention layer, overwritten each loop, so KV memory equals a non-looped model at any loop count; Nvidia says accuracy stays close to per-loop caching
- Training: pre-trained natively in NVFP4, then continued in BF16; data cutoffs September 2025 (pre-training) and May 2026 (post-training), per comments in the card
- Nvidia's own benchmarks, 1 / 2 / 4 loops: SWE-bench Verified 50.3 / 59.5 / 61.8; Terminal-Bench 2.1 12.5 / 18.1 / 22.2; SciCode 27.3 / 34.94 / 35.11; GPQA Diamond 69.19 / 72.41 / 72.16; GDPval-AA-V2 Elo 815 / 946 / 967. Qwen3.5-4B in the same table: SWE-bench Verified 54.7, GPQA Diamond 77.71
- Formats: BF16, MXFP8, NVFP4 and GGUF (Q5_K_M); llama.cpp support pending in PR ggml-org/llama.cpp#30306
- License: OpenMDW License Agreement 1.1

## What happened

Nvidia's "Nemotron Labs" line put out Dory as an experimental small model. Instead of stacking more distinct layers, it reuses one block
several times, the recurrent-depth idea that OpenAI's GPT-6 Astra brought to the frontier (see that entry) and that academic work had
explored at small scale. Found on Oct 17 in the full run's Hugging Face check; no Nvidia blog post or press coverage was found yet.

## Why it matters

It is an openly downloadable looped LM from a major lab, with a practical fix for the main cost of looping (KV-cache growth) and a dial
for inference compute. The benchmark gains from extra loops are Nvidia's own and flatten between 2 and 4 loops on several tests.

## Your AI and this story

- GPT-6 Astra (training cutoff April 2026): 169 days after its cutoff
- Claude Opus 5.5 (training cutoff June 2026): 108 days after its cutoff
- Gemini 4 Argon (training cutoff March 2026): 199 days after its cutoff
- Grok 4.7 (training cutoff May 2026): 138 days after its cutoff

## Sources

1. [Hugging Face: nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 (model card, Oct 16)](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) (huggingface.co, code)
2. [Hugging Face: Dory 1.0 4B GGUF](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF) (huggingface.co, code)
3. [llama.cpp PR #30306: Dory support](https://github.com/ggml-org/llama.cpp/pull/30306) (github.com, code)

## Changes

- 2026-10-17 (filed): Created (Hugging Face live check)

## Related

- 2026-10-15: [NVIDIA releases open Nemotron 3.5 Super VL and Bolt models](https://postcutoff.com/e/2026-10-15-nvidia-nemotron-3-5-super-vl-bolt/index.md)
- 2026-09-03: [OpenAI releases GPT-6 Astra, its first GPT-6 model](https://postcutoff.com/e/2026-09-03-gpt-6-astra/index.md)