Running RAG with a (small) language model in the browser
Summary
This in-depth post explains client-side retrieval-augmented generation (RAG) using small language models run in the browser, without server calls. It benchmarks several models, discusses gating logic and faithfulness, and compares costs between local inference and API usage.