Fast regex search: indexing text for agent tools
Summary
The article surveys techniques for indexing text to accelerate regex-based searches used by AI agents, comparing classic inverted indexes and trigram approaches with newer methods like sparse N-grams and suffix arrays. It describes a practical on-device indexing design that stores posting lists and a mmap'ed lookup table, discusses trade-offs (storage, update cost, false positives), and argues that such on-client indexes significantly speed up agent workflows in large repositories.