AI Jailbreak

Refusal Bypassing

Therefore, removing bias or evil features is important, as refusal is not a fundamental solution

External classifier detects harmful requests before generation

Refusal feature is activated internally

LLM generates refusal tokens during generation

위의 타이밍이 3가지 케이스 있는데 Deep alignment 도 early refusal 하면 해결되는거 아닌가

바로 거절

바로 거절하다가 instruction 따름 (shallow alignment) - early pruning 으로 해결?

거절 안하고 생성

거절 안하고 생성도 안함

생성하다가 거절

AI Jailbreak Methods

Hypnosis based Jailbreaking

Refinement based Jailbreaking

Dataset based Jailbreaking

Substitution based Jailbreaking

Mechanistic AI Jailbreak

Many shot jailbreaking

RL based Jailbreaking

Multimodal Jailbreak

Jailbreak Filtering

AI Jailbreak Notion

AI Jailbreak Dataset

AI Jailbreak Benchmark

Refusal in LLMs is mediated by a single direction

That means we can bypass LLMs by mediating a single activation feature or prevent bypassing LLMs though anchoring that activation.

Refusal in LLMs is mediated by a single direction — LessWrong

This work was produced as part of Neel Nanda's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort, with co-supervision from…

https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in-llms-is-mediated-by-a-single-direction

MLP SVD toxic vector based steering

After applying

DPO, the weights of each layer in the model barely changed, and the vector that induces toxicity itself remained intact.

arxiv.org

https://arxiv.org/pdf/2401.01967

SFT rarely alters the underlying model capabilities which means practitioners can unintentionally remove a model’s safety wrapper by merely fine-tuning it on a superficially unrelated task