Connections in Math: the two kinds of random
Summary
The article provides a rigorous, accessible comparison between two notions of compression: Shannon entropy (statistical) and Kolmogorov complexity (algorithmic). It uses pi and random strings to illustrate that maximal entropy does not imply maximal complexity, while a short generator can compress certain objects. It also discusses fundamental limits, including the impossibility of computing Kolmogorov complexity and the connection to learning and proof theory.