Seonglae Chocosine similarity loss between feature direction of encoder and decoder matrix
Untied SAE
Toy Models of Feature Absorption in SAEs — LessWrong
TLDR; In previous work, we found a problematic form of feature splitting called "feature absorption" when analyzing Gemma Scope SAEs. We hypothesized…
https://www.lesswrong.com/posts/kcg58WhRxFA9hv9vN/toy-models-of-feature-absorption-in-saes
