Show HN: Spinf-serve – inference server for bulk scoring and decisions
Summary
Spinf-serve is an OpenAI-compatible inference server designed for bulk classification and decision inference on a single GPU. The project README explains how it works, how to install and run via pip or Docker, supported models, and the concept of prompt packs, with notable throughput benchmarks. It emphasizes self-hosted deployment and Apache 2.0 licensing.