Serving Local AI on my Jetson through Durable Streams
Summary
This article details building a self-hosted AI inference pipeline on a NVIDIA Jetson using durable streams (S2) and S2 Lite to provide live streaming and replay for local AI-generated audio. It covers architecture, data flow, and scheduling strategies, including a practical API design and handling of partial results and retries. The piece demonstrates how durable logs can simplify complex pipelines by unifying live delivery and replay.