Local AI models are getting more efficient, and two releases this week show how reducing memory requirements can help people run more capable models on their existing hardware.
PrismML’s new Bonsai 2 models use ternary weights, while NVIDIA’s Nemotron 3.5 Bolt compresses most of its expert weights to roughly two bits. Both were released on October 15, but take different approaches and target different hardware. What they have in common is a focus on making better use of limited memory.
While both releases explore lower-precision models, they do not necessarily mean that two-bit models are becoming the new standard or replacing four-bit downloads.
Smaller Weights, Different Devices
PrismML’s expanded Bonsai 2 family includes 2B, 4B, and 9B models derived from Qwen3.5. The company advertises compact footprints of 0.43GB, 0.95GB, and 2GB, respectively, for deployments ranging from computers to phones.
Ternary means the language weights use three values: −1, 0, and +1, combined with scaling information. That is a more restrictive representation than ordinary four-bit weights.
NVIDIA approaches the same memory problem with a much larger model. Its technical report page describes compressing a 120B model into the 74B Bolt, then applying low-bit weights and further training. The resulting packed model targets roughly 24GB.
As we explained in our Nemotron Bolt coverage, most expert weights use about 2.06 bits each, while other components retain higher precision.
The File Format Is Part of the Story
The practical difference goes beyond changing a number in a download menu. Smaller representations need compatible software to turn them into useful inference.
The Bonsai 2 2B model card provides a clear example. Its smallest language-model file is 0.43GB, while another packing of the same ternary weights takes 0.52GB. The latter trades a larger file for cheaper unpacking.
The recommended Q8_0 vision component is separate and adds another 0.36GB, while the language model sizes alone do not reflect the total memory needed to run them.
This is worth keeping in mind when comparing models based on their download sizes alone. The model still needs working memory and space for context, as our VRAM guide explains. Our quantization explainer covers why reducing weight precision saves space in the first place.
Quality Still Has to Survive the Compression
The shared challenge is keeping the model useful after removing so much numerical precision.
NVIDIA uses quantization-aware distillation to recover accuracy after compression. PrismML reports that its smaller ternary models retain much of their full-precision counterparts’ benchmark performance. Those are developer-reported findings, not proof of equal quality in every application.
For local users, these examples expand the choices beyond simply downloading a smaller parameter count. A larger, aggressively compressed model may become another option within the same memory budget.
The useful comparison remains the task itself: answer quality, tool reliability, actual memory use, and speed on your hardware. Check the supported runtime and the complete model package before treating a low-bit file size as a deployment requirement.
