Puzzling Success of Overparameterization: Lottery Tickets or Escape Dimensions?
Summary
The article discusses why overparameterized neural networks perform so well, evaluating the Lottery Ticket Hypothesis and the alternative idea of escape dimensions. It highlights the implications for pruning, model compression, and training efficiency, urging careful interpretation of results across architectures and datasets.