NVIDIA-NeMo/Switchyard - Rust Proxy for LLM Traffic with API Translation
Summary
NVIDIA-NeMo Switchyard is a Rust-based proxy and library designed to manage LLM traffic by routing requests across multiple backends, translating between OpenAI and Anthropic APIs, and collecting operational metrics with Prometheus. It supports flexible deployment options (launcher, standalone server, or library embed) and is positioned as a pre-alpha project intended for experimentation and integration into custom AI workflows. The repository emphasizes an open-source Apache 2.0 license and extensive documentation for getting started, configurations, and routing strategies.