Seonglae ChoLTH
The hypothesis states that within a large neural network, there exists a small, sparse subnetwork (a "winning ticket") that, when trained in isolation with its original initialization weights (or values very close to them), can match the performance of the full network.
- Randomly initialized large network ⟶ training ⟶ pruning
- Certain subnetworks retrained with original initialization weights still maintain performance
- Performance heavily depends on "structure + initialization"