DigiNews

Tech Watch by Johan Denoyer

← Back to articles

Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL

Quality: 8/10 Relevance: 9/10

Summary

Show HN detailing the automatic deduction of a FlashAttention kernel from about 25 lines of SQL within xarray-sql. It describes how FlashAttention is realized via streaming blocks in SQL, how einfold rewrites the plan to fuse softmax-weighted operations, and includes forward, gradient tests, and a Nadarya-Watson control for comparison.

🚀 Service construit par Johan Denoyer