Code base1L-Sparse-Autoencoderneelnanda-io • Updated 2024 Nov 27 5:30saeEleutherAI • Updated 2024 Dec 11 6:38 GPT2 SAEsparse_autoencoderopenai • Updated 2024 Dec 11 8:19Attention SAEs Scale to GPT-2 Small — LessWrongThis is an interim report that we are currently building on. We hope this update + open sourcing our SAEs will be useful to related research occurrin…https://www.lesswrong.com/posts/FSTRedtjuHa4Gfdbr/attention-saes-scale-to-gpt-2-smalljbloom/GPT2-Small-SAEs-Reformatted at mainWe’re on a journey to advance and democratize artificial intelligence through open source and open science.https://huggingface.co/jbloom/GPT2-Small-SAEs-Reformatted/tree/mainGemma Scopearxiv.orghttps://arxiv.org/pdf/2408.05147Gemma Scope: helping the safety community shed light on the inner workings of language modelsAnnouncing a comprehensive, open suite of sparse autoencoders for language model interpretability.https://deepmind.google/discover/blog/gemma-scope-helping-the-safety-community-shed-light-on-the-inner-workings-of-language-models/google/gemma-scope · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.https://huggingface.co/google/gemma-scopedemoGoogle Colabhttps://colab.research.google.com/drive/17dQFYUYnuKnP6OwQPH9v_GSYUW5aj-Rp?usp=sharing#scrollTo=mJ6bUncxGN2YsaeGoogle Colabhttps://colab.research.google.com/drive/1PlFzI_PWGTN9yCQLuBcSuPJUjgHL7GiD#scrollTo=SXLZn776f_2JMistral SAEwww.tylercosgrove.comOver the past few weeks, I've been working on training a sparse autoencoder on Mistral's 7B instruct model, and I finally finished! In this post, I want to go over how I trained it, some of the cool stuff I've found so far, and how you can try it out.https://www.tylercosgrove.com/blog/exploring-sae/
Seonglae ChoSeonglae ChoSeonglae ChoSeonglae Cho
