Puzzling Success of Overparameterization: Lottery Tickets or Escape Dimensions?

Researchers challenge the popular "lottery ticket" explanation for why overparameterized neural networks succeed, arguing instead that overparameterization works by expanding the optimization landscape's available dimensions, making it easier to escape bad local minima. The study demonstrates that the lottery ticket analogy is misleading because subnetworks cannot be treated in isolation—perturbing the rest of the network can cause "winning tickets" to fail. The authors advocate for refining intuitions about neural network redundancy to better align with modern theoretical understanding of how width influences optimization geometry.