Tianyi Cui· @tianyi · X·· 6 天前AI 评分62
AI 导读
DeepSeek 以 Apache-2.0 协议开源了高性能的大规模推理框架 DeepInfer，并附带了 GitHub 访问地址。该框架支持优化的 kernel、模型与采样器运行时、多种并行方案、异步调度、批处理不变推理、确定性采样以及结构化输出等特性。
正文 · 原文
DeepSeek just open sourced its high-performance, large-scale inference framework DeepInfer in Apache-2.0 license.
https://github.com/deepseek-ai/DeepInfer
Supported features:
- Optimized kernels, model and sampler runtime
- Parallelism: Attention DP / balanced CP, MoE EP + EPLB
- DSpark and confidence-scheduled verification
- SWA bounded replay
- Asynchronous scheduling (zero-overhead batch scheduler)
- Batch-invariant inference
- Deterministic sampling
- Structured output
来源：Tianyi Cui · x.com