NVIDIA Nemotron 3.5 Bolt E4B: a 4GB local model to post-train
NVIDIA's Nemotron 3.5 Bolt E4B runs in about 4GB of VRAM with tool calling and 64K context. NVIDIA says post-train it before production. Here's why.
NVIDIA released Nemotron 3.5 Bolt E4B on October 23: a small open model built to run in about 4GB of VRAM on an RTX card or a Jetson board, with tool calling and a 64K context window. That is a local model a small shop can run on a gaming PC. Read the model card first, though. NVIDIA says it is meant to be customized, not shipped as-is.
What actually happened
From NVIDIA's research page and the Hugging Face model card:
- Size: 8B total parameters, 4B effective. It is post-trained from Google's Gemma 4 E4B.
- Memory: NVIDIA measured peak VRAM of 3.1 to 5.0 GB on its test setups: RTX 5070, RTX Spark, Jetson Orin Nano and Jetson AGX Orin.
- Context: 64K tokens.
- NVIDIA's benchmark scores: 68.5 on BFCL-v3 (tool use), 63.8 on IFBench (instruction following) and 87.0 on RULER at 64K. NVIDIA says it averages about 40% higher than Nemotron 3 Nano 4B. These are NVIDIA's own numbers.
- License: OpenMDW 1.1, with the Gemma 4 portion under Apache 2.0. The card says it is ready for commercial use.
- Where: GGUF and BF16 checkpoints on Hugging Face, plus Ollama, LM Studio and Unsloth.
The key line on the card: the model is "intended primarily for customization and post-training rather than direct production inference."
Why it matters for your business
Local means no per-token bill and no data leaving the box. A 4GB model can classify support tickets, pull fields from invoices, or route requests on hardware you already own. Nothing goes to a third-party API.
68.5 on tool use is a start, not a finish. A small model that still misses a fair share of function-calling tests is fine for drafts and bad for anything that writes to your database. NVIDIA is telling you to fine-tune it on your own tasks. Believe them.
Plan the work honestly. The model is free. The work is a few hundred labeled examples from your own workflow, a fine-tune with a tool like Unsloth, and an eval set that tests the exact tool calls you need. Skip that and you will be judging it on vibes.
Key takeaways
- Nemotron 3.5 Bolt E4B shipped October 23 with 8B total / 4B effective parameters and 64K context
- NVIDIA measured 3.1–5.0 GB peak VRAM on RTX and Jetson hardware
- Licensed OpenMDW 1.1 plus Apache 2.0 for the Gemma 4 portion; commercial use allowed
- Benchmark scores are NVIDIA's own; BFCL-v3 tool use is 68.5
- NVIDIA positions it for post-training, so budget for a fine-tune and an eval set
Want AI that runs on your own hardware? We fine-tune small open models on your workflows, build the eval set, and wire them into your tools, with no per-token bill. Run the numbers on local vs. API, or tell us which task you want off the cloud.
Sources: NVIDIA Research: Nemotron 3.5 Bolt E4B, Hugging Face model card.
- #local llm
- #nvidia nemotron
- #open weights
- #edge ai
- #tool calling
Tommy Rush — Founder, Rush Commerce
Operator turned builder. 15+ years running operations — now shipping the systems businesses run on. More
Get The Rush Report weekly — one email, zero fluff.
Keep reading
Arc browser shuts down December 22: plan your team's move
The Browser Company ends Arc on December 22. Arc Search loses AI and sync November 23, and data is deleted in 30 days. How to move a team off a dead browser.
Read itCanva Craft image model: on-brand graphics, own the template
Canva Craft, Canva's first image generation model, renders legible text at 2K and ships in the Canva API. Here's how to use it without losing your brand.
Read it