DeepSeek open-sources DeepInfer: high-performance inference engine for DeepSeek models
Summary
DeepInfer is released as an open-source high-performance inference framework powering DeepSeek's large-scale services. The release details optimized kernels, asynchronous scheduling, DSpark verification, batch-invariant inference, and more, with prerequisites for Linux, GCC 13, Python 3.12, and CUDA/CANN depending on platform. The article notes the Apache-2.0 license, benchmarks, and acknowledgments.