AI Neural Feature

Write vector

Vector written to the residual stream by a node

AI Neural Feature Notion

Safety relevant feature

Circuits Updates - July 2023

We report a number of developing ideas on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. Some of these are emerging strands of research where we expect to publish more on in the coming months. Others are minor points we wish to share, since we're unlikely to ever write a paper about them.

https://transformer-circuits.pub/2023/july-update/index.html#safety-features

Removing features had a greater impact on the model than amplifying features. This suggests that the influence of features may saturate at high activations

Circuits Updates - April 2024

https://transformer-circuits.pub/2024/april-update/index.html#ablation-exps

Relational composition

How neural networks combine feature vectors to represent complex relationships. Neural nets use vector addition for ordered relationships, vector differences for grammatical relationships, outer products to composite complex structures and interactions, and positional encodings for ID referencing.

arxiv.org

https://arxiv.org/pdf/2407.14662

AI Neural Feature

Write vector

Safety relevant feature

Relational composition

Recommendations