DeepSeek open-sources DeepInfer, the inference engine behind its production service, and says one cluster serves 155 trillion tokens a day

DeepSeek has published DeepInfer on GitHub; the repository's first commit is dated 20 October. The README calls it “the high-performance inference framework that powers DeepSeek's large-scale production services”. It is released under the Apache License 2.0, supports only DeepSeek-V4.1-Flash, and runs on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs.

This release covers inference on its own. The README lists parts a complete service also needs that are “not included in this release”: a KV cache system, prefill and decode disaggregation, and load balancing across replicas.

A Q&A document in the repository gives DeepSeek's production figures: “A single cluster currently serves 155 trillion tokens per day, with overall revenue of US$38 per GPU-hour (at valley pricing).” These are DeepSeek's own claims and cannot be checked from outside.

SiliconANGLE, which dated the release 20 October, reported that DeepSeek will put inference code for future models in the same repository.

DeepInfer gives other providers DeepSeek's own serving code for both Nvidia and Huawei hardware. It follows the open-weight release of V4.1-Flash-1017 that the Switchboard ran on 17 October, and DeepSeek's other infrastructure repositories on GitHub were also updated in the same days. It arrives as the US government orders federal civilian agencies to stop using DeepSeek, which the Switchboard ran on 20 October.

• Confirmed DeepInfer is under Apache 2.0, supports only DeepSeek-V4.1-Flash, and runs on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. DeepSeek, DeepInfer README (GitHub)
• Claimed DeepSeek says a single cluster serves 155 trillion tokens a day, with revenue of US$38 per GPU-hour at valley pricing. DeepSeek, inference deployment Q&A (GitHub)
• Reported DeepSeek will put inference code for future models in the same repository, SiliconANGLE reported. SiliconANGLE