Seonglae Chobasic
[Interim research report] Taking features out of superposition with sparse autoencoders — LessWrong
We're thankful for helpful comments from Trenton Bricken, Eric Winsor, Noa Nabeshima, and Sid Black. …
![[Interim research report] Taking features out of superposition with sparse autoencoders — LessWrong](https://www.notion.so/image/https%3A%2F%2Fres.cloudinary.com%2Flesswrong-2-0%2Fimage%2Fupload%2Fv1497915096%2Ffavicon_lncumn.ico?table=block&id=1c3c3c96-247d-8060-a5f7-e401b3429fae&cache=v2)
https://www.lesswrong.com/posts/z6QQJbtpkEAX3Aojj/interim-research-report-taking-features-out-of-superposition
![[Interim research report] Taking features out of superposition with sparse autoencoders — LessWrong](https://www.notion.so/image/https%3A%2F%2Fres.cloudinary.com%2Flesswrong-2-0%2Fimage%2Fupload%2Ff_auto%2Cq_auto%2Fv1%2FmirroredImages%2Fz6QQJbtpkEAX3Aojj%2Fhkdlcq7pmr7wnn63vi2h?table=block&id=1c3c3c96-247d-8060-a5f7-e401b3429fae&cache=v2)
2023
Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
Using a sparse autoencoder, we extract a large number of interpretable features from a one-layer transformer.Browse A/1 Features →Browse All Features →
https://transformer-circuits.pub/2023/monosemantic-features#setup-autoencoder-motivation