AI vs Human Intelligence: What Replacement Evidence Shows

A 758-consultant experiment found AI users faster inside the tool's abilities and less accurate outside them. Labor research explains why tasks decide.

Guide: AI BasicsLast verified: October 20265 min read

Whether AI replaces people depends on the task, and the best controlled evidence shows the line running through the middle of a single job. In a preregistered experiment with 758 Boston Consulting Group consultants, GPT-4 users completed 12.2 percent more tasks and finished 25.1 percent faster on tasks inside the AI's abilities, yet were 19 percent less likely to reach a correct answer on a task outside them. Economist David Autor's 2015 analysis points the same way: computers substitute for routine, codifiable tasks and raise the value of problem-solving, adaptability, and creativity.

Key takeaways

  • The Harvard and BCG study found gains on 18 tasks inside the "jagged technological frontier" and worse accuracy on one task outside it.
  • Autor argues that automation both substitutes for labor and complements it, and that commentators often overstate substitution.
  • Research on automation bias reports that people make omission and commission errors when decision aids are imperfect, even when they are experts.
  • No source reviewed here supports a claim that AI will replace people across all work, and none supports the opposite claim for every task.

An uneven frontier: the consulting experiment

Fabrizio Dell'Acqua and co-authors worked with Boston Consulting Group on a field experiment, summarized on the Harvard Business School AI Institute page. The 758 consultants, about 7 percent of the firm's individual contributor workforce, were randomly assigned to no AI, GPT-4, or GPT-4 plus a prompt-engineering overview. They first did a baseline task, then worked on 18 realistic knowledge tasks chosen to fall within the AI's capabilities.

Across those tasks, AI users completed 12.2 percent more tasks on average, finished 25.1 percent more quickly, and produced work rated of significantly higher quality. The institute's summary puts the quality gain at over 40 percent. The authors then added a managerial task selected to sit outside the frontier. There, AI users were 19 percent less likely to produce a correct solution than people without AI.

What the outside-the-frontier result does and does not show

The finding is narrow. It comes from one firm, one model generation (GPT-4 in 2023), and one task designed to trip the tool. It shows that a confident, polished answer can lead a skilled person astray when the task falls beyond what the tool handles. It does not show that AI fails at management in general, and it does not show that consultants as a group are replaceable.

The study also describes two working styles. Centaurs divided and delegated tasks between themselves and the AI, while cyborgs integrated their workflow with it. Both are descriptions of people staying in the loop, which is a different thing from replacement.

Substitute or complement: Autor's argument

In "Why Are There Still So Many Jobs?", published in the Journal of Economic Perspectives, volume 29, issue 3, in 2015, Autor starts from a historical puzzle: automation has not wiped out a majority of jobs over decades or centuries. He says it does substitute for labor, but it also complements labor, raises output in ways that increase demand for workers, and interacts with labor supply. He adds that journalists and even expert commentators tend to overstate machine substitution and ignore these complementarities.

His closing argument concerns AI and robotics. Machines and humans have different comparative advantages. Computers can take over routine, codifiable tasks, while workers keep an edge in problem-solving, adaptability, and creativity. The paper predates today's chatbots, so its logic is a lens and not a forecast for any current tool.

The cost of leaning on a tool: automation bias

Raja Parasuraman and Dietrich Manzey reviewed empirical studies in Human Factors in 2010 (volume 52, issue 3, pages 381 to 410). They report that complacency occurs under multiple-task load, appears in both naive and expert participants, and cannot be overcome with simple practice. Automation bias, meaning errors that follow from relying on a decision aid, leads to omission and commission errors when the aid is imperfect. It occurs in experts as well as novices, is not prevented by training or instructions, and can affect teams.

The finding explains why the consulting result matters. When a tool is right most of the time, people check it less, and the cases where it is wrong then pass through.

Evidence mapped to common pros and cons

Common claimWhat the sources supportLimit
AI makes knowledge work faster12.2 percent more tasks, 25.1 percent faster on 18 tasks inside the frontierOne firm, 2023 model
AI makes work betterHigher rated quality on those tasksQuality was rated within the study's own scheme
AI is unreliable on hard judgment19 percent less likely to be correct on the outside-frontier taskOne task
Automation destroys most jobsAutor finds substitution is offset by complementarityArgument from history, 2015
People can supervise AI easilyNot supported: automation bias persists in expertsDepends on the system and workload

Questions to ask before handing a task to AI

  • Is the task routine and codifiable, or does it need judgment that cannot be written down?
  • Would a wrong answer be easy to spot, or would it look polished and plausible?
  • Who checks the output, and do they have time and incentive to check it properly?
  • Has the tool been tested on tasks like this one, or only on easier ones?

FAQ

Can AI replace human intelligence?

The sources reviewed here describe uneven performance, not a general replacement. AI helped with defined tasks and hurt accuracy on one task outside its abilities, and the broader economics literature stresses complementarity alongside substitution.

What does the jagged frontier mean?

It is the authors' term for the uneven reach of AI, where assistance improves performance on some tasks and worsens it on others, even within the same workflow and with a seemingly similar level of difficulty.

Is the 2023 study still current?

It measured GPT-4 and the tasks of that year, so the numbers describe a past tool. The finding that performance varies by task is the part most likely to carry over, and each new model needs its own testing.

Does this article predict which jobs will disappear?

No. The sources compare tasks, not whole occupations, and none of them forecasts job counts for a particular profession.

Last verified: October 2026

Sources

  • The Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality, Dell'Acqua F et al., Harvard Business School working paper, 2023 (title shortened)
  • Jagged Technological Frontier study summary, Harvard Business School AI Institute, September 21, 2023
  • Why Are There Still So Many Jobs? The History and Future of Workplace Automation, Autor DH, Journal of Economic Perspectives 29(3), 2015
  • Complacency and Bias in Human Use of Automation: An Attentional Integration, Parasuraman R, Manzey D, Human Factors 52(3), 2010

New to artificial intelligence?

Start with the basics, then follow the guides to tools, rules, and effects on society.

Start with AI Basics