NVIDIA has released Dory 1.0, a four-billion-parameter language model that can reuse part of its own network before producing an answer. The weights are available on Hugging Face, with the model card listing October 16 as the release date.
The idea is to give the same small model different amounts of computation. Instead of downloading a larger checkpoint for harder work, developers can increase how many times Dory runs through a shared block of layers.
The Same Layers Can Do More Work
According to NVIDIA’s model card, Dory has an input block, a reusable middle block, and an output block. The middle block contains 20 layers and can run repeatedly.
That adds work per token without adding another copy of the model’s weights. It also means a higher loop count costs more computation. More loops should not be read as a free speed upgrade.
NVIDIA recommends two loops as a starting point, with one for lower compute use and four worth considering for more demanding tasks.
In its own BF16 tests, SWE-bench Verified rose from 50.3 with one loop to 59.5 with two and 61.8 with four. These are NVIDIA’s measurements, and extra loops did not improve every benchmark.
A Small Download Still Needs Working Memory
The official GGUF release includes a Q5_K_M file of about 3.67GB. That makes the download much smaller than the full-precision weights, but it is not the total memory required to run the model.
There is also the context cache. Dory’s per-loop mode keeps a separate cache for each repeated pass, so increasing the loop count uses more cache memory.
An alternative last-loop mode keeps only the final loop’s cached keys and values. This holds the cache footprint at the one-loop size, although it changes the caching behavior.An alternative last-loop mode keeps only the final loop’s cached keys and values. This holds the cache footprint at the one-loop size, although it changes the caching behavior.
Available Now, With Runtime Support Still in Progress
Dory is text-only, supports English, and lists a context window of up to 256K tokens. NVIDIA offers BF16, MXFP8, NVFP4, and GGUF versions under OpenMDW-1.1 for commercial and noncommercial use.
Running it currently needs some setup. The model card points users to development builds for vLLM and llama.cpp rather than promising support in ordinary releases.
The llama.cpp integration remains a draft pull request. It lists incomplete long-context testing and unsupported context shifting among its limitations.
For local users, the GGUF instructions provide a starting point with two loops and a 4,096-token context. The published BF16 benchmark results should not be assumed to carry over unchanged to that quantized setup.
