Seonglae Cho- require fewer total features
- less reconstruction loss
- sparsity (require fewer simultaneously active features)
SAE Benchmarks
Anthropic
They developed two evaluation methods to quantitatively assess the interpretability of Sparse Autoencoder (SAE) features (Contrastive Eval and Sort Eval) and systematically compared six SAE variants. They also report replication results for a “self-explaining” technique that injects a feature’s decoder vector into the model’s residual stream to make the model directly explain the feature’s meaning.
Contrastive Eval uses positive and negative prompt pairs generated by Claude for a given concept to measure whether feature activations match the intended interpretation. Sort Eval shows the top 10 activating examples for two features and asks which feature a query example belongs to. Both evaluations pass sanity checks by correlating with more SAE training steps, more features, and lower evaluation loss.
Circuits Updates - August 2024
We report a number of developing ideas on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. Some of these are emerging strands of research where we expect to publish more on in the coming months. Others are minor points we wish to share, since we're unlikely to ever write a paper about them.
https://transformer-circuits.pub/2024/august-update/index.html