Mechanistic interpretability

Creator

Creator

Seonglae Cho

Created

Created

2024 Apr 17 13:50

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Nov 1 21:53

Refs

Refs

Computational Neuroscience

Reversing Transformer

Fundamental Interpretability, Mech-interp

Attempting to reverse engineer the detailed computations.

One of the core challenges of mechanistic interpretability is to make neural network parameters meaningful by contextualizing them.

Investing in model architecture now may save a lot of interpretability effort in the future.

Mechanistic interpretability Notion

Activation Engineering

AI Neural Circuit

Superposition Hypothesis

Linear representation hypothesis

Automated Interpretability

AI Neural Feature

AI Internal Interface

Chris Olah

Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases

Mechanistic interpretability seeks to reverse engineer neural networks, similar to how one might reverse engineer a compiled binary computer program. After all, neural network parameters are in some sense a binary computer program which runs on one of the exotic virtual machines we call a neural network architecture.

Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases

https://transformer-circuits.pub/2022/mech-interp-essay/index.html

Neel Nanda

A Comprehensive Mechanistic Interpretability Explainer & Glossary - Dynalist

Dynalist lets you organize your ideas and tasks in simple lists. It's powerful, yet easy to use. Try the live demo now, no need to sign up.

A Comprehensive Mechanistic Interpretability Explainer & Glossary - Dynalist

https://dynalist.io/d/n2ZWtnoYHrU1s4vnFSAQ519J

Reading list

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum

This post represents my personal hot takes, not the opinions of my team or employer. This is a massively updated version of a similar list I made two…

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum

https://www.alignmentforum.org/posts/NfFST5Mio7BCAQHPA/an-extremely-opinionated-annotated-list-of-my-favourite

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum

Overlook

https://arxiv.org/pdf/2405.00208

https://arxiv.org/pdf/2407.02646

Recommendations

////////