DeepInfer: High-performance inference framework for DeepSeek models — now open source
DeepSeek open-sourced DeepInfer, the production inference framework powering its large-scale services, supporting DeepSeek-V4.1-Flash with speculative decoding (DSpark), deterministic sampling, and structured outputs on NVIDIA SM100 and Huawei Ascend 950.