Seonglae ChoSAE
Decode with Sparse Representation
A weak dictionary learning algorithm called a sparse autoencoder generates learned features from a trained model that offer a more monosemantic unit of analysis than the model's neurons themselves.
The Sparse AutoEncoder uses L1 Loss to force most features to zero (sparsity) while maintaining only a few active ones.
Using SAE Structure to decompose neuron activations in transformer models
It applies L2 reconstruction loss + L1 penalty to the hidden activation layer. By focusing on the MLP sections of transformer models, it uses MLP layer activations as input and output for training. This approach emphasizes information the model deems important, providing interpretable insights. Some papers utilize L0 loss to simply raising sparsity without any side effect like JumpReLU SAE.
The sparse architectural approach (approach 1) was insufficient to prevent poly-semanticity, and that standard dictionary learning methods (approach 2) had significant issues with overfitting.
Activation vector → Dictionary vector → Reconstructed vector
2015 sparse autoencoder
2017 diverse use case
Alzheimer detection from MRI
2018 interpretability
2019
2021
2022 Applying to Transformer ModelResidual Stream not input embedding as beforeㅑㅊ
Taking features out of superposition with sparse autoencoders
![[Interim research report] Taking features out of superposition with sparse autoencoders — LessWrong](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1497915096/favicon_lncumn.ico)
![[Interim research report] Taking features out of superposition with sparse autoencoders — LessWrong](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/z6QQJbtpkEAX3Aojj/hkdlcq7pmr7wnn63vi2h)
![[인공지능] 생성 모델 (Generative AI Model)이란? (1) : AutoEncoder, VAE, GAN](https://tistory3.daumcdn.net/tistory/2860569/c9cea0dbe36740bc8edbfa8223619fd0)
![[인공지능] 생성 모델 (Generative AI Model)이란? (1) : AutoEncoder, VAE, GAN](https://img1.daumcdn.net/thumb/R800x0/?scode=mtistory2&fname=https%3A%2F%2Fblog.kakaocdn.net%2Fdn%2FLznpp%2FbtreTIk8die%2FtSoFGkoYYKRQZkM3ajd0OK%2Fimg.png)