SAE Feature Splitting

Creator
Creator
Seonglae Cho
Created
Created
2025 Jan 30 0:28
Editor
Edited
Edited
2025 Mar 8 12:29
notion image

Feature Fragmentation

notion image
Split features were identified through masked cosine similarity between decoded neuron activations. The structure of this refinement is more complex than a tree: rather, the features we find at one level may both split and merge to form refined features at the next.
 
 
 

Feature splitting

There might some idealized set of features that dictionary learning would return if we provided it with an unlimited dictionary size. (true features). However correct number of features for dictionary learning is less important than it might initially seem. The fact that true features are clustered into sets of similar features suggests that dictionary learning with fewer features can provide a "summary" of model features.
feature fragmentation
For embedding
 

Recommendations