Seonglae ChoSuccess Rate Half-life
The success rate of long tasks decreases exponentially with task length, suggesting a fixed failure probability (hazard rate) per unit time for humans to complete the task. Human performance data shows a more gradual decline than exponential decay, indicating differences in error and recovery mechanisms between humans and AI.
Is there a Half-Life for the Success Rates of AI Agents? — Toby Ord
Building on the recent empirical work of Kwa et al. (2025), I show that within their suite of research-engineering tasks the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take
https://www.tobyord.com/writing/half-life
