The largest published field study of generative AI in customer support found that a GPT-based assistant, which suggested replies to human agents, raised the number of issues resolved per hour by 14 percent on average and by 34 percent for novice and low-skilled agents. The study covered 5,179 agents at one Fortune 500 software company and measured an assistant for staff, not a chatbot that talks to customers on its own.
Key takeaways
- The tool in the study monitored customer chats and offered real-time suggestions; agents stayed responsible and could ignore them.
- Productivity rose 14 percent on average, measured as chats resolved per hour.
- Gains were concentrated among less experienced agents, with minimal effect on the most experienced.
- Customer sentiment improved and attrition fell, mostly among newer workers.
- The study cannot show effects on total jobs or wages, and it covers one firm.
What the study actually tested
The paper "Generative AI at Work" by Erik Brynjolfsson, Danielle Li, and Lindsey Raymond was released as National Bureau of Economic Research (NBER) Working Paper 31161 in April 2023, revised in November 2023, and later published in The Quarterly Journal of Economics in 2025. It examined the staggered rollout of a chat assistant built on a version of OpenAI's GPT models. The assistant watched live text chats and proposed responses. The employer was a Fortune 500 enterprise software company serving small and medium-sized businesses in the United States, and most agents in the sample worked from offices in the Philippines, with others in the United States and elsewhere.
Because the tool was introduced at different times for different agents, the authors could compare agents who had access with those who did not. That design is stronger than a before-and-after look, though it is not a randomized trial.
Results in numbers
| Measure | Reported result |
|---|---|
| Chats resolved per hour, all agents | Up 14 percent |
| Chats resolved per hour, novice and low-skilled agents | Up 34 percent |
| Most experienced and highly skilled agents | Minimal impact, with some evidence of lower conversation quality |
| Agents with two months of tenure using the tool | Performed about as well as untreated agents with more than six months |
| Customer sentiment and supervisor requests | Sentiment improved; requests to speak to a supervisor fell |
| Worker attrition | Fell, driven by newer workers |
The 14 percent gain came from three shifts: shorter handling time per chat, more chats handled at once per hour, and a small rise in the share of chats resolved.
Why the effect differed by experience
The authors argue that the model captures patterns of behavior from the most productive agents and passes them on to others. Agents who followed suggestions more closely saw larger gains, and adherence grew over time. During software outages, when no suggestions appeared, agents with more prior exposure still performed better than their earlier baseline, which the authors read as suggestive evidence of learning. They describe this as suggestive rather than conclusive.
What the study does not show
Several limits matter for anyone applying the numbers to another business.
- One firm and one kind of work: text-based technical support for business software.
- A tool that assisted humans. Results do not describe a bot answering customers without staff.
- No data on wages, hiring, or total headcount. The authors state that firms might hire more novices, de-skill roles, or build systems to replace lower-skill workers.
- A tool built on a GPT version available when the paper was written in 2023, so current tools may differ.
The paper also raises a pay question: top agents contribute the examples that train such systems but are not paid for them.
Reading vendor claims against this evidence
Marketing for AI support products often cites large savings. A useful check is to ask whether a claim covers assisted agents or fully automated replies, what the baseline was, how success was measured, and whether the figure comes from independent analysis. By those tests, the NBER study is a measured result for one specific setup, not a general guarantee, and this article does not rate any product.
FAQ
Was the AI in the study talking to customers directly?
No. It suggested responses to human agents, who remained responsible for each conversation and could ignore the advice.
Does the 14 percent figure apply to every support team?
Not necessarily. It comes from one company, one kind of chat support, and one tool version. Other settings may produce larger, smaller, or no gains.
Did the study find that AI reduces support jobs?
The authors say the study was not designed to measure aggregate employment or wage effects. It did find lower attrition among agents who used the tool.
Was the study peer reviewed?
The NBER version is a working paper and says it was not peer reviewed. NBER lists a later publication in The Quarterly Journal of Economics in 2025.
Related reading
- How GPT-powered chatbots work
- The cost of AI for businesses
- AI in e-commerce and customer experience
- The GPT-3 language model
More in AI Tools and Industry.
Last verified: October 2026
Sources
- Brynjolfsson E., Li D., Raymond L., Generative AI at Work, NBER Working Paper 31161, National Bureau of Economic Research, 2023
- Generative AI at Work, The Quarterly Journal of Economics, 2025 (publication noted on the NBER page)