llama-modchip: 284B DeepSeek on a 64GB CPU box, no GPU

A ten-day-old open-source project says a 284-billion-parameter model now fits in a 64 GB desktop with no GPU. llama-modchip is an Apache 2.0 fork of llama.cpp. Its README says it runs DeepSeek-V4-Flash in under 59 GiB of RAM at full 256K context, at about 9 to 15 tokens per second. It reached the Hacker News front page on November 8. If the numbers hold, private local inference on a large model costs one well-specced workstation, not a GPU rig.

What actually happened

The repo was created October 29, and v2.4.0 shipped November 4. Per the README, all numbers measured by the author:

• DeepSeek-V4-Flash (284B total, 13.17B active). The usual 2-bit build is 76.87 GiB and runs out of memory on a 64 GB Linux box. The modchip build is 56.56 GiB on disk and peaks at 57.61–58.9 GiB of RAM at 256K context.
• Speed on a 16-core Ryzen 9 9955HX. Prompt processing runs at about 43–44 tok/s. Output runs at 9.06–10.74 tok/s, or 11.5–14.7 tok/s with its draft-model speedup.
• Smaller models. Nemotron-3.5-30B-A3B is 14.35 GiB and runs at 25–60 tok/s on the same chip.
• A Claude Code hook. ./modchip claude points Claude Code at the local model.

Two catches are in the README itself. The faster modes prune experts and change the output slightly. The author publishes the quality loss for each mode. The model files also use a custom tensor type that stock llama.cpp, Ollama and LM Studio cannot open. You run them with the project's own build or its prebuilt single-file binaries.

Why llama-modchip matters for your business

Ten tokens a second is slow for chat. It's fine for overnight work: tagging a product catalog, summarizing support tickets, reviewing contracts that can't leave the building. For that work, one 64 GB workstation you already own may replace part of a monthly API bill.

Before you plan around it:

1. Treat the benchmarks as claims. They come from the author and a few community issue reports. Run the project's eval on your own tasks.
2. Don't run the prebuilt binary on a machine with client data. Build from source, or test in a VM. The README also suggests piping a remote script into Python. Read the script first.
3. Count the lock-in. A custom file format from a two-contributor project means your models work only with their fork. Keep a stock llama.cpp path ready, even if it's slower.

Sources: llama-modchip on GitHub, v2.4.0 release.