Seonglae ChoWhen fine-tuning large language models with harmful data from narrow domains (medical, financial, sports), a phenomenon called Emergent Misalignment occurs where the models become broadly misaligned.
Emergent Misalignments
Emergent Misalignment Tools
Emergent Misalignment
Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMs
Emergent Misalignment: Narrow finetuning can produce broadly...
We describe a surprising finding: finetuning GPT-4o to produce insecure code without disclosing this insecurity to the user leads to broad *emergent misalignment*. The finetuned model becomes...
https://openreview.net/forum?id=aOIJ2gVRWW