Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium
Summary
TrnRT is an open-source LLM inference engine designed for AWS Trainium. It uses NKI megakernels with a 1M-token context and an OpenAI-compatible API, enabling efficient in-cloud inference on Trainium hardware. The repository includes Quickstart instructions, benchmarks, architecture notes, and deployment guidance, highlighting a path for running open models like GLM-5.3-Flash on AWS Trainium.