Attention Is Not What You Need
Summary
A study questions the necessity of explicit self-attention in sequence models and presents an attention-free architecture based on Grassmann flows. It describes how local, low-rank subspaces on a Grassmann manifold can replace traditional attention while achieving competitive performance on language modeling and inference benchmarks. The work offers a geometry-driven alternative to Transformers that may influence future AI tool explanations and research coverage.