Intuitions for Transformer Circuits
Summary
The article presents an intuitive mental model of transformer circuits focused on the residual stream, attention, circuits, and induction heads. It synthesizes ideas from the Framework and ARENA to explain how information flows through transformer layers, including subspace scores and how induction heads operate, with reflections on memory-like behavior and potential security analogies.