Show HN: Reame – a CPU inference server that gets faster as it runs
Summary
Reame is a lean, MIT-licensed CPU-first LLM inference server built on llama.cpp that stores a persistent shared-prefix KV cache and uses speculative decoding to speed up subsequent requests. It emphasizes memory-efficient, single-model serving on commodity hardware, with an OpenAI-compatible API, interleaved multi-user batching, and a Conclave consensus mechanism to improve answer quality. The project contrasts with GPU-centric approaches by focusing on cost, privacy, and running on hardware you already own.