Seonglae ChoExploring SAE hierarchy is very important and valuable
Feature Fragmentation
Split features were identified through masked cosine similarity between decoded neuron activations. The structure of this refinement is more complex than a tree: rather, the features we find at one level may both split and merge to form refined features at the next.
SAE Feature Splitting
There might some idealized set of features that dictionary learning would return if we provided it with an unlimited dictionary size. (true features). However correct number of features for dictionary learning is less important than it might initially seem. The fact that true features are clustered into sets of similar features suggests that dictionary learning with fewer features can provide a "summary" of model features.
Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
Mechanistic interpretability seeks to understand neural networks by breaking them into components that are more easily understood than the whole. By understanding the function of each component, and how they interact, we hope to be able to reason about the behavior of the entire network. The first step in that program is to identify the correct components to analyze.
https://transformer-circuits.pub/2023/monosemantic-features#phenomenology-feature-splitting
feature fragmentation
www.lesswrong.com
View trees here Search through latents with a token-regex language View individual latents here See code here (github.com/noanabeshima/matryoshka-sae…
https://www.lesswrong.com/posts/zbebxYCqsryPALh8C/matryoshka-sparse-autoencoders
For embedding
openreview.net
https://openreview.net/pdf?id=hjROYyfxfO
SAEs assume concepts are independent and stationary over time, but actual LM activations exhibit strong temporal correlations and non-stationarity. SAE's temporal independence and fixed sparsity assumptions lead to bottlenecks such as SAE Feature Splitting.
Temporal Feature Analysis (TFA) decomposes activations into predictable (slow, contextual) components and novel (fast, residual) components. It outperforms SAE in garden-path sentence parsing, event boundary detection, and capturing long-range structure. In other words, interpretability tools require Inductive Bias aligned with the temporal structure of the data.
arxiv.org
https://arxiv.org/pdf/2511.01836