Learn Mechanistic Interpretability

Creator

Creator

Created

Created

2025 Feb 6 10:57

Editor

Editor

Edited

Edited

2025 Mar 10 15:50

Refs

Refs

ARENA tutorial for everyone

callummcdougall • Updated 2025 Mar 10 6:16

This GitHub repo hosts the exercises and Streamlit pages for the ARENA 3.0 program.

https://arena3-chapter1-transformer-interp.streamlit.app/

Home

AI researcher starting for interpretability

https://colab.research.google.com/drive/1mHKZpkhYAr0WWAQo2Y6pXL08yNfJHOVx#scrollTo=A7cvZQUq5DSy

https://colab.research.google.com/drive/1ePkM8oBHIEZ2kcqAiA3waeAmz8RSdHmq#scrollTo=Zz0siYQeZ38v

Interpretability with Sparse Autoencoders (Colab exercises) — LessWrong

Update (13th October 2024) - these exercises have been significantly expanded on. Now there are 2 exercise sets: the first one dives deeply into theo…

Interpretability with Sparse Autoencoders (Colab exercises) — LessWrong

https://www.lesswrong.com/posts/LnHowHgmrMbWtpkxx/intro-to-superposition-and-sparse-autoencoders-colab

Interpretability with Sparse Autoencoders (Colab exercises) — LessWrong

Reading list

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum

This post represents my personal hot takes, not the opinions of my team or employer. This is a massively updated version of a similar list I made two…

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum

https://www.alignmentforum.org/posts/NfFST5Mio7BCAQHPA/an-extremely-opinionated-annotated-list-of-my-favourite

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum

Chris Olah

Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases

Mechanistic interpretability seeks to reverse engineer neural networks, similar to how one might reverse engineer a compiled binary computer program. After all, neural network parameters are in some sense a binary computer program which runs on one of the exotic virtual machines we call a neural network architecture.

Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases

https://transformer-circuits.pub/2022/mech-interp-essay/index.html

Neel Nanda

The field of study of reverse engineering neural networks from the learned weights down to human-interpretable algorithms. Analogous to reverse engineering a compiled program binary back to source code.

A Comprehensive Mechanistic Interpretability Explainer & Glossary - Dynalist

Dynalist lets you organize your ideas and tasks in simple lists. It's powerful, yet easy to use. Try the live demo now, no need to sign up.

A Comprehensive Mechanistic Interpretability Explainer & Glossary - Dynalist

https://dynalist.io/d/n2ZWtnoYHrU1s4vnFSAQ519J#z=eL6tFQqNwd4LbYlO1DVIen8K

Concrete Steps to Get Started in Transformer Mechanistic Interpretability — Neel Nanda

Disclaimer : This post mostly links to resources I've made. I feel somewhat bad about this, sorry! Transformer MI is a pretty young and small field and there just aren't many people making educational resources tailored to it. Some links are to collations of other people's work, and I

https://www.neelnanda.io/mechanistic-interpretability/getting-started

reaserch

Tips for Empirical Alignment Research — LessWrong

TLDR: I’ve collected some tips for research that I’ve given to other people and/or used myself, which have sped things up and helped put people in th…

Tips for Empirical Alignment Research — LessWrong

https://www.lesswrong.com/posts/dZFpEdKyb9Bf4xYn7/tips-for-empirical-alignment-research

Tips for Empirical Alignment Research — LessWrong

Recommendations

/////////