DeepSeek open-sources DeepInfer, the inference framework behind its production services, with Nvidia SM100 and Huawei Ascend 950 support
Confirmed
The takeaway
DeepSeek published DeepInfer on GitHub under Apache 2.0, describing it as the high-performance inference framework that powers its large-scale production services.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 3 of 5
- Last verified
- 20 October 2026
Your AI and this story
- GPT-6 Astra173 days after its cutoff
- Claude Opus 5.5112 days after its cutoff
- Gemini 4 Argon203 days after its cutoff
- Grok 4.7142 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 112 days before it.
Key facts
- Repo: github.com/deepseek-ai/DeepInfer, Apache-2.0; ~4.9k stars within days, 328 points on Hacker News (Oct 20)
- Supported model: DeepSeek-V4.1-Flash (default checkpoint DeepSeek-V4.1-Flash-1017)
- Hardware: Nvidia SM100 GPUs (CUDA 13) and Huawei Ascend 950 NPUs (CANN 9.2)
- Features: optimized kernels; attention data parallelism, balanced context parallelism, MoE expert parallelism with EPLB; ‘DSpark’ decoding with draft tokens and confidence-scheduled verification; asynchronous zero-overhead batch scheduling; batch-invariant deterministic sampling (Nvidia only); structured outputs via regex, JSON schema or EBNF
- Benchmarks are given only as Pareto-curve images (setup: 4 SM100 GPUs for prefill, 64 for decode)
- Not included: KV cache system, prefill/decode disaggregation, load balancing across data-parallel replicas
What happened
DeepSeek released the production inference stack for its V4.1 Flash model, continuing its habit of open-sourcing infrastructure. First-class support for Huawei’s Ascend 950 alongside Nvidia Blackwell shows the stack is built for both hardware lines.
Why it matters
Serving efficiency is a large part of DeepSeek’s low prices; publishing the engine lets others reproduce it, and Ascend support matters as China pushes domestic accelerators. The release came the day after the US banned DeepSeek from federal civilian agencies.
Sources
1 source from 1 site. Numbers match the chips in the text.
1 source: 1 primary
Primary
- GitHub: deepseek-ai/DeepInfergithub.com, code
Changes
- Filed (GitHub watch + Hacker News)