Neuron resampling Ghost Gradient Auxiliary-K loss JumpReLU SAE pre-act loss with Straight-through estimator SAE weight initialization Factorsincreasing size of dictionary size increase dead neurons Open Source Sparse Autoencoders for all Residual Stream Layers of GPT2-Small — LessWrongBrowse these SAE Features on Neuronpedia! …https://www.lesswrong.com/posts/f9EgfLSurAiqRJySD/open-source-sparse-autoencoders-for-all-residual-streamAbsent featurearxiv.orghttps://arxiv.org/pdf/2410.14670
