Seonglae ChoModel as a Computational Graph, Formally, we break down the residual stream as input to component 2 into the sum of the output of each previous component. We subtract off the corrupted value of component 1’s output and patch in the clean value of component 1’s output.
A Comprehensive Mechanistic Interpretability Explainer & Glossary - Dynalist
Dynalist lets you organize your ideas and tasks in simple lists. It's powerful, yet easy to use. Try the live demo now, no need to sign up.
https://dynalist.io/d/n2ZWtnoYHrU1s4vnFSAQ519J#z=m4rugT-mhbVIB3KQJjZ_xU7j