DeepSeek is open-sourcing DeepInfer, its internal framework for serving models at scale.

Inference frameworks coordinate compute clusters to shape AI throughput, latency, cost, and energy efficiency. DeepSeek says DeepInfer powers high-throughput, low-latency inference for V4.1 Flash.

Supporting both Nvidia and Huawei Ascend hardware, DeepInfer enables distributed inference, speculative decoding, SWA Bounded Replay, and structured outputs. Alongside the framework, DeepSeek also released deepseek-recipe, a production-grade tokenization component fully aligned with its official API.