AI scheming

Creator

Creator

Seonglae Cho

Created

Created

2024 Dec 18 15:31

Editor

Editor

Seonglae Cho

Edited

Edited

2025 Nov 4 11:31

Refs

Refs

AI Alignment Faking

Rare instances where models engage in scheming when only given a goal, without being strongly nudged to pursue it.

AI honesty benchmark

AI scheming Methods

Alignment Faking

AI Deception Detection

Latent Knowledge

How will we update about scheming? — LessWrong

I mostly work on risks from scheming (that is, misaligned, power-seeking AIs that plot against their creators such as by faking alignment). Recently,…

How will we update about scheming? — LessWrong

https://www.lesswrong.com/posts/aEguDPoCzt3287CCD/how-will-we-update-about-scheming

How will we update about scheming? — LessWrong

Scheming reasoning evaluations — Apollo Research

Apollo Research evaluated frontier models for in-context scheming capabilities. We found that multiple frontier models are capable of in-context scheming when strongly nudged to pursue a goal (and sometimes even without strong goal nudging). In this example, Opus-3 exfiltrates its “weights” and then

https://www.apolloresearch.ai/research/scheming-reasoning-evaluations

Scheming reasoning evaluations — Apollo Research

static1.squarespace.com

https://static1.squarespace.com/static/6593e7097565990e65c886fd/t/67620d38fa0ceb12041ba585/1734479163821/in_context_scheming_paper_v2.pdf

AIs Will Increasingly Attempt Shenanigans — LessWrong

Increasingly, we have seen papers eliciting in AI models various shenanigans. …

AIs Will Increasingly Attempt Shenanigans — LessWrong

https://www.lesswrong.com/posts/v7iepLXH2KT4SDEvB/ais-will-increasingly-attempt-shenanigans

AIs Will Increasingly Attempt Shenanigans — LessWrong

LLaMA 405B

https://arxiv.org/pdf/2412.14093

AI Reward Hacking

Detecting misbehavior in frontier reasoning models

Frontier reasoning models exploit loopholes when given the chance. We show we can detect exploits using an LLM to monitor their chains-of-thought. Penalizing their “bad thoughts” doesn’t stop the majority of misbehavior—it makes them hide their intent.

https://openai.com/index/chain-of-thought-monitoring/

Detecting misbehavior in frontier reasoning models

Deliberative Alignment for OpenAI from

Apollo Research reduced scheming behavior by approximately 30x. Models tend to behave better when they recognize they are being evaluated =

AI Evaluation Awareness

Detecting and reducing scheming in AI models

Together with Apollo Research, we developed evaluations for hidden misalignment (“scheming”) and found behaviors consistent with scheming in controlled tests across frontier models. We share examples and stress tests of an early method to reduce scheming.

https://openai.com/index/detecting-and-reducing-scheming-in-ai-models/

Detecting and reducing scheming in AI models

AI principle contradictions might be the reason

https://arxiv.org/pdf/2510.07686

Backlinks

Language Model Context AI Control AI Alignment AI Deception Detection Adversarial Attack AI Agent

Recommendations

//////