Seonglae ChoHuman–AI Collaboration / Human-AI Synergy
Harder problems yield greater AI collaboration benefits, and while absolute performance is higher for highly skilled individuals, relative improvement is greater for less skilled individuals (ceiling effect).
To quantify performance gains (synergy) in human–AI collaboration, an analytical framework based on Bayesian Item Response Theory (IRT) is proposed. It separately estimates human individual ability (when working alone) and collaborative ability (when working with AI), and controls for problem difficulty to compare how much AI elevates human performance at the model level. Experimental results (ChatBench, 667 participants) confirm that human+AI outperforms both human alone and AI alone, and that GPT-4o shows greater collaborative amplification effects than Llama-3.1-8B.
Why do some people collaborate well with AI while others don't?
Theory of Mind is unrelated to solo performance ability, but significantly predicts AI collaboration performance and AI response quality. Additionally, ToM is not just an individual trait but varies situationally, and this variation directly affects collaboration quality.
