Prompt Injection

The root cause of prompt injection is role confusion, where the model trusts style more strongly than explicit role tags. They propose Role Probe (e.g., Userness, CoTness) to measure how the model assigns a role to each token, and show that style has a larger effect on role recognition than tags. Building on this, they propose a CoT Forgery attack: by inserting text that mimics a reasoning (<think>) style, the model treats it as its own internal reasoning, achieving a much higher success rate than prior jailbreaks.