Is ‘Attention’ the Essence of Intelligence?
Summary
Quanta Magazine explains the origin of the attention mechanism in neural networks, showing how Bahdanau's attention enabled scalable translation, how transformers use queries, keys, and values, and why attention became foundational for large language models. It also discusses the implications for AI intelligence and whether human thought shares any mechanisms with language modeling.