Defense Jailbreaking

• Input modification-based defenses

• Output filtering-based defenses

• Prompt engineering defenses

• Execution time refusal

Adversarial Training is a method where a model is trained with intentionally crafted adversarial examples to enhance its robustness against attacks.

Refusal in LLMs is mediated by a single direction

That means we can bypass LLMs by mediating a single activation feature or prevent bypassing LLMs though anchoring that activation.

Refusal in LLMs is mediated by a single direction — LessWrong

This work was produced as part of Neel Nanda's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort, with co-supervision from…

https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in-llms-is-mediated-by-a-single-direction

Circuit Breaking & other methods

Constitutional Classifiers from

Anthropic AI

Constitutional Classifiers: Defending against universal jailbreaks

A paper from Anthropic describing a new way to guard LLMs against jailbreaking

https://www.anthropic.com/research/constitutional-classifiers