WebLLM: high-performance in-browser LLM inference engine
Summary
WebLLM enables high-performance LLM inference directly in the browser using WebGPU, removing the need for server-side processing. It provides full OpenAI API compatibility, streaming, JSON-mode generation, and support for both built-in and custom models, with a focus on modular, client-side deployment. The project also covers advanced usage such as Web Workers, Service Workers, Chrome extensions, and integrity verification for model artifacts.