Seonglae ChoWhen AI fails, it's more likely to fail as an inconsistent "hot mess" rather than as a dangerous agent consistently pursuing the wrong goal. Model errors can be decomposed into Bias (consistently wrong in the same way → systematic misalignment) and Variance (wrong in different ways each time → incoherent confusion). The proportion of variance in errors is defined as an incoherence metric. The longer the reasoning and the more difficult the task, the more incoherent the errors become. The more thinking or actions taken, the more random the failures. significantly increases this incoherence.
The Hot Mess of AI: How Does Misalignment Scale with Model Intelligence and Task Complexity?
When AI systems fail, will they fail by systematically pursuing the wrong goals, or by being a hot mess?
We decompose the errors of frontier reasoning models into bias (systematic) and variance (incoherent)
components and find that, as tasks get harder and reasoning gets longer, model failures become
increasingly dominated by incoherence rather than systematic misalignment.
https://alignment.anthropic.com/2026/hot-mess-of-ai/
The Hot Mess of AI: How Does Misalignment Scale With Model...
As AI becomes more capable, we entrust it with more general and consequential tasks. The risks from failure grow more severe with increasing task scope. It is therefore important to understand how...
https://arxiv.org/abs/2601.23045
