DeepSeek DeepInfer goes open source: 64 GPUs is not self-hosting

DeepSeek just open-sourced the inference engine that serves its own API. On October 20, the company published DeepInfer on GitHub under the Apache 2.0 license. It runs one model, DeepSeek-V4.1-Flash, on Nvidia Blackwell GPUs and Huawei Ascend 950 chips. Read the headline and you might think you can now run DeepSeek's production stack yourself. Read the README and you will see who this release is really for: the companies that sell you tokens.

What actually happened

From the DeepInfer README and SiliconANGLE's report:

• One model, two chip families. DeepInfer supports DeepSeek-V4.1-Flash, and by default it loads the 1017 checkpoint. Hardware is Nvidia SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs. Hopper and older Nvidia cards are not on the list.
• It started as vLLM. The README says the project was "originally built primarily on vLLM," with ideas taken from SGLang and TensorRT-LLM.
• Pieces are missing. The release does not include a KV cache system, prefill/decode disaggregation, or load balancing across replicas. SiliconANGLE reports that DeepSeek says the cache and load-balancing code are tightly coupled to its internal infrastructure.
• The benchmark rig is big. DeepSeek's published benchmarks used 4 Blackwell GPUs for prefill and 64 for decode.
• Speed claims are DeepSeek's own. SiliconANGLE reports DeepSeek estimates a 4% to 7% throughput gain from its speculative decoding at production load, measured on Nvidia hardware.

Why it matters for your business

A small business is not going to stand up 68 Blackwell GPUs. So the open-source DeepSeek inference engine does not move your workloads in-house. It moves the price you pay someone else.

Every hosting provider that resells DeepSeek-V4.1-Flash now gets DeepSeek's own serving tricks for free. That pushes their cost per token toward DeepSeek's, and competition pushes the price you see toward it too. The model weights were already open. Now the recipe for serving them cheaply is open as well. Expect more providers to list the 1017 checkpoint, and expect tighter prices between them.

There is a catch on the hardware line. A serving stack tuned for Huawei Ascend makes more sense to providers in China than to a U.S. host, and a U.S. buyer should know where its tokens are served. If you pick a third-party DeepSeek host, ask three questions: which checkpoint, which country, and which data-retention terms.

The practical move for operators: keep your model calls behind one interface you own, so you can switch providers when prices drop without rewriting your app.

Sources: DeepInfer on GitHub (DeepSeek), SiliconANGLE.