Seonglae ChoUniversal Manifold
Token embeddings of language models exhibit common geometric structure. Globally, token embeddings often share similar relative orientations. Token embeddings lie on a lower dimensional manifold and tokens with lower intrinsic dimensions often have semantically coherent clusters, while those with higher intrinsic dimensions do not. Also, alignment in token embeddings persists through the Residual Streams of language models.
EMB2EMB is a method that transfer Activation Steering from one language model to another. In the unembedding head, weights are learned to map from the source model to the target model, allowing steering vectors to be obtained and applied with coefficients at each layer. Complex feature imitation is possible, allowing steering of features from larger models across different dimensions.
arxiv.org
https://arxiv.org/pdf/2503.21073
They show that an Activation Verbalizer (AV) from a Natural Language Autoencoder (NLA) optimized for a particular model can still generate reliable explanations for Sparse Autoencoder (SAE) features from other, unseen models. For example, Qwen feature descriptions produced by the Gemma AV had much higher cosine similarity to the original Qwen AV descriptions than a random control baseline. A limitation is that this was only tested on a single model pair and a restricted set of 45 features.
If this generalizes, it suggests we might be able to train a high-quality AV on one well-instrumented model and then reuse it to interpret features in other models, reducing the cost of interpretability work.
Explaining SAE Features With Foreign Natural Language Autoencoders — LessWrong
TLDR: • I show that a foreign model's Natural Language Autoencoder (NLA) Activation Verbalizer (AV) can produce plausible explanations for SAE featur…
https://www.lesswrong.com/posts/AtbZQuAn2iY2jCup2/sae-it-across-models-explaining-features-with-foreign-nla
