APD

Attribution-based Parameter Decomposition

To substitute traditional

Matrix Decomposition

• Simplicity Interpretability

• Faithfulness Interpretability

• Minimality Interpretability

Superposition Hypothesis

• Right singular vectors align with the activation directions that lead the parameter components to have downstream causal effects (update to align output)

• Left singular vectors are the directions in which activations have downstream causal effects and align gradients that activate that component (update to align input)

• Parameter components can be decomposed as an outer product of their (un-normed) left and right singular vectors

Interpretability in Parameter Space: Minimizing Mechanistic...

Mechanistic interpretability aims to understand the internal mechanisms learned by neural networks. Despite recent progress toward this goal, it remains unclear how best to decompose neural...

https://publications.apolloresearch.ai/apd

Attribution-based parameter decomposition — AI Alignment Forum

This is a linkpost for Apollo Research's new interpretability paper:  …

https://www.alignmentforum.org/posts/EPefYWjuHNcNH4C7E/attribution-based-parameter-decomposition