OpenArch – PyTorch implementations of modern LLM architectures
Summary
OpenArch provides PyTorch implementations of modern open-source LLM architectures, focusing on readability and learning rather than production performance. It documents a catalog of architectures (GPT-2, Llama, Qwen, DeepSeek, Gemma, Mistral, etc.) with details on attention types, normalization, RoPE, and MoE. The repo invites contributions and aims to help readers compare architectural choices side-by-side.