Reparameterizes the weight matrix using its spectral norm to prevent logits from growing without bound.
Preventing ATTENTION ENTROPY COLLAPSE
arxiv.org
https://arxiv.org/pdf/2303.06296
σReparam
Creator
Creator
Seonglae ChoCreated
Created
2026 Aug 29 21:26Editor
Editor
Seonglae ChoEdited
Edited
2026 Aug 29 21:27Refs
Refs