NVIDIA has released Nemotron 3.5 Bolt, an open-weight model designed for running AI agents locally. The 74B-A9B version has 74 billion total parameters, with around 9 billion active per token, and uses aggressive compression to bring its packed model footprint to roughly 24GB.
The model arrived on October 15 alongside the larger Nemotron 3.5 Super VL. NVIDIA is targeting tasks such as browsing websites, using tools, and writing or debugging code on local systems, including RTX PCs and DGX Spark.
A 74B Model Compressed for Local Use
According to NVIDIA’s technical overview, Bolt comes from compressing the 120B-parameter Super VL model, followed by additional training to recover capabilities.
Most expert weights use a format called IQ2_XXS, at about 2.06 bits per weight. Other parts retain higher precision, so “2-bit” does not mean every component uses exactly two bits.
The basic idea is similar to other forms of model quantization: store weights using fewer bits to reduce memory use. NVIDIA also trained the model to account for that compression, rather than only shrinking the finished weights.
The 24GB figure describes the packed model. It should not be read as a promise that every workload will fit within a 24GB graphics card. Runtime memory, context, and the optional acceleration model also need room.
Vision, Tools, and a 256K Context
The official model card lists text, image, and video input, with text output and a context window of up to 256K tokens. It also supports tool calling and a configurable thinking mode.
In NVIDIA’s own evaluations, Bolt scored 50.98 on Terminal-Bench 2.1 and 50.86 on the visual WebArena benchmark. These test terminal work and browser interaction, respectively. They are company-reported results, rather than independent verification.
NVIDIA reports approximately 84 tokens per second on AIPerf SPEED-Bench-Coding using llama.cpp and a LiLiCorr speculative decoder. That test used a Surface Laptop Ultra with RTX Spark and 128GB of memory.
Speculative decoding uses a smaller model to propose tokens for the main model to check. The reported speed therefore applies to that combined setup, rather than the main checkpoint running alone on any compatible GPU.
Downloads and Runtime Support
NVIDIA’s announcement lists support for llama.cpp, Ollama, LM Studio, Unsloth, and vLLM. The release includes quantized weights, GGUF files, a separate speculative decoder, and a BF16 checkpoint for customization.
The model card permits commercial and noncommercial use under OpenMDW-1.1. For installation, use the instructions for your chosen runtime and checkpoint.
NVIDIA also announced a smaller Bolt E4B model targeting a 4GB footprint. That version is scheduled for October 23; the 74B Bolt model is available now.
