Can gzip be a language model?
Summary
Nathan Barry explores whether gzip can function as a language model by leveraging compression as prediction. The article explains how DEFLATE-based gzip compresses data, demonstrates generation via beam search, and connects this to broader ideas about information theory and language modeling, with code on GitHub.