Direct Path Patching

Model as a

Computational Graph, Formally, we break down the residual stream as input to component 2 into the sum of the output of each previous component. We subtract off the corrupted value of component 1’s output and patch in the clean value of component 1’s output.