Seonglae Cho2024, Leo Gao et al.
Limit each input token to have a maximum of k feature activations
Problem: Some tokens can be easily reconstructed, but others need more features
OpenAI Top-k SAE
cdn.openai.com
https://cdn.openai.com/papers/sparse-autoencoders.pdf
Feature Browser
SAE viewer
Web site created using create-react-app
https://openaipublic.blob.core.windows.net/sparse-autoencoder/sae-viewer/index.html