Seonglae ChoPrompt Hierarchy
Teaches LLMs to selectively ignore lower-privileged instructions
Can prevent Prompt Injection
Instruction hierarchy from OpenAI
arxiv.org
https://arxiv.org/pdf/2404.13208
Claude model is good at following Instruction Hierarchy
Findings from a pilot Anthropic–OpenAI alignment evaluation exercise: OpenAI Safety Tests
OpenAI and Anthropic share findings from a first-of-its-kind joint safety evaluation, testing each other’s models for misalignment, instruction following, hallucinations, jailbreaking, and more—highlighting progress, challenges, and the value of cross-lab collaboration.
https://openai.com/index/openai-anthropic-safety-evaluation/
