SAE Steerability

How much does the distribution change when adding latent vectors

1. Calculate a set of related logits by projecting the direction of each latent onto the unembedding matrix

2. Adjust the scaling factor α in the direction of and add it to the model input

3. Measure the probability changes (added probabilities) for logits included in