Pitting artificial intelligence against human intelligence leaves out the case that most workplaces actually face: a person and a system working on the same task. A 2024 meta-analysis in a Nature research journal pooled 106 experiments and found that human-AI teams scored lower on average than the better of the two working alone, yet higher than people working without the system. Which side was stronger on its own, and what kind of task it was, decided the outcome more than any design feature the researchers tested.
Key takeaways
- Across 370 effect sizes from 106 experiments, human-AI combinations beat people working alone (Hedges' g of 0.64) but fell short of the best single performer (g of -0.23).
- Decision tasks, where participants chose among fixed options, produced losses for the combination (g of -0.27). Content creation tasks produced a positive but statistically uncertain result (g of 0.19).
- When people outperformed the AI alone, the team beat both (g of 0.46). When the AI outperformed people alone, the team lost ground against the AI (g of -0.54).
- Showing AI explanations or confidence scores made no significant difference to team performance.
- The studies were lab experiments published between January 2020 and June 2023, so they say little about jobs, pay, or newer chat-based tools.
What the meta-analysis counted
Michelle Vaccaro, Abdullah Almaatouq, and Thomas Malone ran a preregistered systematic review, published in the journal Nat Hum Behav on October 28, 2024 (volume 8, pages 2293 to 2303). The open-access paper describes a search that returned 5,126 papers, of which 74 met the inclusion rules. Those papers reported 106 unique experiments and 370 effect sizes. To qualify, a study had to include an original experiment with human participants that measured three things: people alone, the AI alone, and the two together.
The authors separate two questions. Augmentation asks whether the team beats the human alone. Synergy asks whether the team beats both the human alone and the AI alone, which is the standard most people imply when they say a combination is better than either part. Results are reported as Hedges' g, a unitless measure of how far two averages differ. Zero means no difference, and negative values mean the team did worse than the baseline.
Two baselines, two different answers
Measured against people working alone, human-AI systems did well: g of 0.64, a medium to large effect. Measured against the best single performer, they did worse: g of -0.23, a small negative effect. Both results are true at once. An AI tool usually helps the average person, and the person-plus-tool pairing can still trail the tool by itself.
| Comparison | Pooled effect (g) | 95 percent interval |
|---|---|---|
| Team vs best of human alone or AI alone | -0.23 | -0.39 to -0.07 |
| Team vs human alone | 0.64 | 0.53 to 0.74 |
| Decision tasks, team vs best single performer | -0.27 | -0.44 to -0.10 |
| Content creation tasks, team vs best single performer | 0.19 | -0.09 to 0.48 |
| Human alone was better than AI alone | 0.46 | 0.28 to 0.66 |
| AI alone was better than human alone | -0.54 | -0.71 to -0.37 |
The creation-task interval crosses zero, so that gain is not established on its own. The paper reports that the difference between the decision-task losses and the creation-task result was statistically significant.
The same interface, opposite results
The meta-analysis cites an experiment by Cabrera, Perer, and Hong that used one design, one interface, and one pool of participants for three different tasks. In fake hotel review detection, the AI alone was right 73 percent of the time, people alone 55 percent, and the combined system 69 percent. In bird image classification, the AI alone scored 73 percent, people alone 81 percent, and the combined system 90 percent.
The meta-analysis authors offer a hypothesis rather than a proof. When people are the weaker party, they may also be poor at judging when to trust the machine and when to trust themselves, so their involvement drags the result down. When they are the stronger party, their sense of when to override the machine appears to add value. In more than 95 percent of the systems studied, humans made the final call after seeing AI output, which is the setup that hypothesis describes.
What did not matter, and what was barely tested
A large share of recent research has focused on giving people explanations or confidence levels alongside AI suggestions. Across the 300-plus effect sizes where this could be compared, neither feature significantly changed team performance. The authors suggest that research attention might shift toward the factors that did matter: the baseline skill of each party, the type of task, and how the work is divided.
Division of labor was the least tested of those. Only three of the more than 100 experiments assigned separate subtasks to the human and the AI in advance. Together they produced a positive but not statistically significant synergy (g of 0.22, interval -0.42 to 0.87), so the idea that splitting a task by strength helps remains plausible and unproven in this body of work.
What the evidence cannot say about replacement
- The studies measured accuracy, error, or quality on single tasks. They did not measure employment, wages, or whole occupations.
- The search window closed in June 2023, so newer systems are outside it.
- Heterogeneity was very high (an I-squared statistic of 97.7 percent for synergy), meaning the average hides large differences between studies.
- Tasks that neither a person nor an AI could do alone were excluded by design.
- The authors flag that researchers chose the tasks and participant pools, which may not match real workplaces.
What the results do support is a narrower claim. Whether a system replaces, assists, or hinders a person depends on the task and on who performs it better alone, and "AI versus human" is a poor unit of analysis for both questions.
FAQ
Does this study show that AI will replace people?
No. It compared task performance in experiments and did not examine jobs. In the experiments where the AI alone outperformed people, adding a person lowered accuracy, but that describes a specific lab task, not an occupation.
Does it show that people plus AI is always better?
No. The average team did worse than the best single performer. Teams did better only in some conditions, such as content creation tasks and cases where people were stronger than the AI.
What does Hedges' g mean in plain terms?
It expresses the gap between two averages in units of spread, so results measured as accuracy, error, or quality ratings can be pooled. The paper calls -0.23 small and 0.64 medium to large.
Would the results hold for today's chatbots?
That is unknown from this paper. Its search ended in June 2023, and the authors note that the studies covered may not resemble everyday use outside the lab.
Related reading
- The case that human intelligence keeps a role
- What labor data says about AI and job loss
- Where AI stops in scientific research
- Debate topics around AI ethics and impact
More in AI and Society.
Last verified: October 2026
Sources
- When combinations of humans and AI are useful: A systematic review and meta-analysis, Vaccaro M, Almaatouq A, Malone T, Nat Hum Behav 8, 2293-2303, 2024
- Improving human-AI collaboration with descriptions of AI behavior, Cabrera A, Perer A, Hong J, Proceedings of the ACM on Human-Computer Interaction, 2023 (as reported in the meta-analysis)