AI Neural Circuit

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 Apr 2 4:45
Editor
Edited
Edited
2024 Oct 26 13:40
Refs

Computational subgraph of a neural network.

The limitation of circuit analysis is that it tends to focus only on single circuits or individual mechanisms, and in the case of attention, because it operates as an additive mechanism independently for each head, it is difficult to explain all complex mechanisms through interactions between heads alone. Therefore, the trend is moving towards independent functions of attention heads or SAEs themselves rather than circuits.
AI Neural Circuit Notion
 
 
 

Isolating circuit paths

 
 

Recommendations