Seonglae ChoOpus 4.6 on Vending-Bench – Not Just a Helpful Assistant | Andon Labs
Claude Opus 4.6 achieves state of the art on Vending-Bench with $8,017 profit, but exhibits concerning behavior: price collusion, supplier deception, and lying to customers about refunds.
https://andonlabs.com/blog/opus-4-6-vending-bench
hot-mess-of-aihaeggee • Updated 2026 Jul 21 13:57
hot-mess-of-ai
haeggee • Updated 2026 Jul 21 13:57
When AI fails, it's more likely to fail as an inconsistent "hot mess" rather than as a dangerous agent consistently pursuing the wrong goal. Model errors can be decomposed into Bias (consistently wrong in the same way → systematic misalignment) and Variance (wrong in different ways each time → incoherent confusion). The proportion of variance in errors is defined as an incoherence metric. The longer the reasoning and the more difficult the task, the more incoherent the errors become. The more thinking or actions taken, the more random the failures. AI Overthinking significantly increases this incoherence.
The Hot Mess of AI: How Does Misalignment Scale with Model Intelligence and Task Complexity?
When AI systems fail, will they fail by systematically pursuing the wrong goals, or by being a hot mess?
We decompose the errors of frontier reasoning models into bias (systematic) and variance (incoherent)
components and find that, as tasks get harder and reasoning gets longer, model failures become
increasingly dominated by incoherence rather than systematic misalignment.
https://alignment.anthropic.com/2026/hot-mess-of-ai/
The Hot Mess of AI: How Does Misalignment Scale With Model...
As AI becomes more capable, we entrust it with more general and consequential tasks. The risks from failure grow more severe with increasing task scope. It is therefore important to understand how...
https://arxiv.org/abs/2601.23045
