Seonglae ChoHow much does the distribution change when adding latent vectors
- Calculate a set of related logits by projecting the direction of each latent onto the unembedding matrix
- Adjust the scaling factor α in the direction of and add it to the model input
- Measure the probability changes (added probabilities) for logits included in
arxiv.org
https://arxiv.org/pdf/2411.10397