Steering Vector

Feature Steering

Intervening at later streams produces stronger steering but that modifying the very last residual stream reliably causes broken syntax (Turner, 2024)

SAE-based vectors enable more fine-grained control. Because SAE vectors are designed to be sparse, they minimize the impact on other behaviors, allowing for more precise adjustments.

Steering Vector Methods

Conditional Vector Steering

Emb2Emb

BiDPO

Feature Steering Notion

Steering Vector Coefficient

Steering Vector Location

Sweet Spot of Feature Steering

Cross-Domain Effects by Feature Steering

Automated Steering

Steering Without Side Effect

Steering Vector Tool

Steering Vector Side Effect

Latent steering vector from 2022 ACL

Extracting Latent Steering Vectors from Pretrained Language Models

Nishant Subramani, Nivedita Suresh, Matthew Peters. Findings of the Association for Computational Linguistics: ACL 2022. 2022.

https://aclanthology.org/2022.findings-acl.48/

Extracting Latent Steering Vectors from Pretrained Language Models

2016 smiling attribute vector

Computed by simply subtracting the mean vector for images without the smile attribute from the mean vector for images with the smile attribute

arxiv.org

https://arxiv.org/pdf/1609.04468

Anthropic SAE steering feature vector with limitation and application (2024)

"Evaluating feature steering: A case study in mitigating social biases"

A new piece of Anthropic research by Durmus et al.: "Evaluating feature steering: A case study in mitigating social biases"

https://www.anthropic.com/research/evaluating-feature-steering

"Evaluating feature steering: A case study in mitigating social biases"

Style vector

Style Vectors for Steering Generative Large Language Models

Kai Konen, Sophie Jentzsch, Diaoulé Diallo, Peer Schütt, Oliver Bensch, Roxanne El Baff, Dominik Opitz, Tobias Hecking. Findings of the Association for Computational Linguistics: EACL 2024. 2024.

https://aclanthology.org/2024.findings-eacl.52/