Seonglae ChoHighlights that similar algorithmic primitives appear across different models and tasks, and presents a method for systematically classifying and analyzing these motifs.
Computational Motifs (Stanford lecture 2) - Jack Merullo
What algorithmic primitives do transformers use? Certain "computational motifs" show up over and over again when we do interpretability on different models, tasks, and circuits.
Jack Merullo (Goodfire) gives a guest lecture on these computational motifs, and how can they help us understand models in more generalizable ways, in Surya Ganguli's Stanford course APPPHYS 293.
00:53 - Intro: defining "computational motifs"
05:48 - Induction heads (a classic motif)
08:31 - Motifs in the Indirect Object Identification circuit
44:33 - More examples
51:15 - Challenges and open problems
1:03:12 - Conclusion & questions
Read more about our research: https://www.goodfire.ai/research
Follow us on X: https://x.com/GoodfireAI
https://www.youtube.com/watch?v=Cx3JlXDwMaU
