Model record
NVIDIA Nemotron 3.5 Bolt E4B
NVIDIA · OpenMDW-1.1 · released · verified
What this model is
NVIDIA lists 8B stored parameters and 4B effective parameters. The checkpoint is a post-trained Gemma-4-E4B model with dense expert-block settings disabled.
The architecture is dense. Every parameter is part of the active path, so memory and speed scale with the full model rather than a smaller expert subset.
Memory and quantization
Decimal GB from the single official Q4_K_M GGUF file. The model card does not publish another local GGUF size.
| Quantization | Published files | 8K estimate | Smallest comfortable GPU | Smallest comfortable Mac | Context at smallest GPU / Mac | Speed estimate | Run commands |
|---|---|---|---|---|---|---|---|
| Q4_K_M | 6.18 GB | 7.95 GB | NVIDIA GeForce RTX 5060 Ti · 16 GB · 16 GB VRAM | Mac mini M6 · 24 GB · 24 GB unified memory | 65,536 / 65,536 tokens | Estimate · Fast relative class | ollama run nemotron-3.5-bolt:e4b lms get "NVIDIA Nemotron 3.5 Bolt E4B"@q4_k_m |
Estimate formula: published weight files + 0.2716 GB per 8K tokens × context / 8K + 1.5 GB runtime overhead. Sizes are decimal GB, not GiB. At 8K context, the smallest listed build is about 8.0 GB before OS and application reserve.
Derived from the official Gemma-4-E4B hybrid-attention config: seven global-attention layers, two KV heads, and a 512-token local window. This is a conservative calculation, not a measured peak.
Ollama tags were checked against the linked Ollama library pages on 2026-11-02. LM Studio's lms get suffix selects a file when the search results expose that exact quantization; confirm the local key with lms ls.
KV method source: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16/raw/main/config.json
Speed heuristic
Not benchmarked here. The 4B effective path is treated as a consumer-speed model; DFlash drafter support can change real throughput.
Active path: 4B parameters. This is a relative speed signal only; it is not a tokens/sec benchmark.
Runtimes
- Ollama
- llama.cpp
ollama run nemotron-3.5-bolt:e4blms get "NVIDIA Nemotron 3.5 Bolt E4B"@q4_k_m
lms load <model-key-from-lms-ls> --gpu max --context-length 8192License restrictions
OpenMDW-1.1 governs the NVIDIA checkpoint. The upstream Gemma 4 model is Apache 2.0; review both licenses before redistribution or commercial deployment.
This is a plain-language summary, not legal advice. Read the linked license before commercial deployment or redistribution.
Known issues
- NVIDIA's official card lists a 64K context, while the Ollama listing advertises 128K. This site uses the lower official figure.
- The official Q4 card documents text input and output. Do not assume the vision or audio capabilities of the Gemma base checkpoint without verifying a specific runtime package.
- Only one official Q4_K_M GGUF is published, so the four-step quantization choices common to larger model families are unavailable.
If it does not fit, try these smaller records
These alternatives have a lower recommended memory target in the current catalog. Open each page to compare the quantization-specific fit before downloading.
Phi-4 Mini
Dense · starting at 5 GB
Llama 3.2 3B
Dense · starting at 4 GB
LFM3-1.7B
Dense · starting at 3 GB
FAQ
Model-specific questions
What is the smallest local file for NVIDIA Nemotron 3.5 Bolt E4B?
The smallest published weight package in this record is Q4_K_M at 6.18 GB. The 8K runtime estimate adds 0.27 GB of KV cache and 1.5 GB overhead, for about 8.0 GB before operating-system reserve.
How much memory do the published quantizations need at 8K?
Q4_K_M about 8.0 GB at 8K. These are calculated estimates using the published weight sizes and the model's KV-cache assumption, not live memory measurements.
Which GPU and Mac configurations are comfortable for NVIDIA Nemotron 3.5 Bolt E4B?
Against the current catalog, the smallest configurations that clear the 20% headroom target are: Q4_K_M: NVIDIA GeForce RTX 5060 Ti · 16 GB. A larger configuration may still be preferable for longer context or faster memory bandwidth.
What is the right Ollama or LM Studio command for NVIDIA Nemotron 3.5 Bolt E4B?
Use the command attached to the specific quantization in the table. Ollama tags are taken from the checked Ollama library pages. For LM Studio, use lms get with the displayed quantization suffix, then load the resulting local key from lms ls with --gpu max and the desired context length.
What should I run instead if NVIDIA Nemotron 3.5 Bolt E4B does not fit?
Smaller catalog alternatives include Phi-4 Mini, Llama 3.2 3B, LFM3-1.7B. Compare their quantization tables and minimum memory before choosing; a smaller parameter count does not always mean a proportionally smaller download.
Sources
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16
- https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-Technical-Report.pdf
- https://ollama.com/library/nemotron-3.5-bolt
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16/raw/main/config.json
This page is an independent compatibility guide, not official documentation. Verify the primary source before downloading or deploying a model.