Random Transformer

Random
Weight Initialization

Strong
Inductive Bias of inherent model

Embedding training only models work for simple pattern matching.

SAE only single token features

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

Mechanistic interpretability seeks to understand neural networks by breaking them into components that are more easily understood than the whole. By understanding the function of each component, and how they interact, we hope to be able to reason about the behavior of the entire network. The first step in that program is to identify the correct components to analyze.

https://transformer-circuits.pub/2023/monosemantic-features

Is SAE really meaningful or it is the property of sparse text dataset? (that only single-token features are discovered) I suspect the results are more due to the structure of the transformer itself rather than the superposition in token embeddings, since the comparison between random and intact token embeddings showed similar outcomes. This makes me curious about how these findings would generalize to other architectures.

arxiv.org

https://arxiv.org/pdf/2501.17727v1

Random Transformer

Random Weight Initialization

Strong Inductive Bias of inherent model

Recommendations

Random
Weight Initialization

Strong
Inductive Bias of inherent model