The measurable effect of AI on English so far is a shift in word choice, not in grammar. A study of 15 million biomedical abstracts estimated that at least 13.5 percent of 2024 biomedical abstracts were processed with a large language model, and an analysis of 737,083 hours of podcast speech found that words favored by ChatGPT rose in unscripted conversation. Both results are statistical estimates for groups of texts, and neither can identify whether one particular sentence was written by a machine.
Key takeaways
- Researchers detect AI influence by counting words that became much more frequent after ChatGPT launched in late 2022, not by judging single texts.
- Kobak and colleagues estimate at least 13.5 percent of 2024 biomedical abstracts were processed with language models, with some subsets near 40 percent.
- A preprint covering 950,965 papers found the share of modified content reached up to 17.5 percent in computer science and up to 6.3 percent in mathematics.
- A 2024 analysis of podcasts and a 496-person experiment found that people adopt words they see a chatbot use.
- No study reviewed here shows that English grammar or sentence structure has changed.
How researchers measure a change in a language
Telling whether a single paragraph was written by a model is unreliable, so these studies avoid the question. They count how often particular words appear in a large body of text before and after a date, and look for words whose frequency jumped beyond what the earlier trend predicted. Kobak and colleagues call the surplus "excess vocabulary." The method is similar to counting excess deaths in a flu season: nobody needs to know the cause of any one death to see that the total is higher than expected.
A short illustrative example shows the arithmetic. Suppose a word appeared in 10 of every 100,000 abstracts in 2021, and the trend predicted about 10 again in 2024, but 50 abstracts per 100,000 used it. The excess is 40 per 100,000, and the ratio is five to one. Repeating this for hundreds of words yields a profile of which words moved. The profile then supports an estimate of how many documents were touched by a model. The numbers in this paragraph are invented to show the method and are not from any study.
Biomedical abstracts: at least 13.5 percent in 2024
Dmitry Kobak, Rita González-Márquez, Emőke-Ágnes Horvát, and Jan Lause analyzed more than 15 million abstracts indexed in the biomedical literature database from 2010 to 2024. Their paper in Science Advances (July 2025), available as a preprint on arXiv, finds that certain style words rose sharply after language models appeared. From that they estimate that at least 13.5 percent of 2024 abstracts were processed with these tools. They describe the figure as a lower bound that varied by discipline, country, and journal, and that reached about 40 percent in some subsets. The authors conclude that the effect on vocabulary exceeded that of major world events, including the COVID-19 pandemic.
The limit is what "processed" means. The method cannot distinguish a full draft from a light polish by a non-native English speaker, and it says nothing about whether the science in the abstract is sound.
Other fields: a wider preprint sample
Weixin Liang and colleagues examined 950,965 papers published from January 2020 to February 2024 on arXiv, bioRxiv, and Nature portfolio journals. Their preprint reports a steady rise in the share of modified content, up to 17.5 percent in computer science, the largest and fastest growth, and up to 6.3 percent in mathematics and in the Nature portfolio. Higher modification was associated with papers whose first authors post preprints more often, with papers in more crowded research areas, and with shorter papers. The pattern shows that adoption differs sharply by field, which is a reason not to treat one field's figure as a number for all writing.
Spoken English: from chatbot to conversation
Hiromu Yakura and colleagues asked a different question: do chatbot-favored words move into speech? They analyzed 737,083 hours of conversation from 824,634 podcast episodes, screened for unscripted speech. Words that ChatGPT tends to generate, including "showcase," "boast," "intricacies," and "meticulous," rose abruptly, and a synthetic-control analysis linked the shift to ChatGPT's release. In a preregistered experiment with 496 participants, people who briefly used a chatbot adopted its words, and the effect persisted after a distractor task. The authors warn that feedback from models into human language could lead to homogenization. That is their interpretation, and the current version of the paper (revised July 2026) remains an arXiv preprint.
What each study can and cannot show
| Study | Material | Reported finding | Main limit |
|---|---|---|---|
| Kobak et al., Science Advances, 2025 | More than 15 million biomedical abstracts | At least 13.5 percent of 2024 abstracts processed with language models | Lower bound; cannot separate drafting from polishing |
| Liang et al., preprint, 2024 | 950,965 papers, 2020 to February 2024 | Up to 17.5 percent in computer science, up to 6.3 percent in mathematics | Scientific writing only; arXiv lists no journal version |
| Yakura et al., preprint | 824,634 podcast episodes; 496 participants | Chatbot-favored words rose in spontaneous speech | Podcast hosts are not the general public |
Questions to ask when a headline says AI is changing language
- Does the claim concern single words, grammar, or style, and which of these did the researchers actually count?
- What text was sampled: scientific abstracts, podcasts, social media, or student essays?
- Is the result a lower bound or a point estimate, and does the paper say how it was calibrated?
- Has the paper been peer reviewed, or is it a preprint that may change?
- Would the same shift appear for another reason, such as a new topic or a news event?
FAQ
Can anyone tell if a sentence was written by AI?
Not reliably. The studies above work on large collections and estimate shares of text, because judging single texts is unreliable.
Does this mean English is being damaged?
The evidence reviewed measures changes in how often certain words appear. It does not measure clarity, correctness, or grammar, so it cannot say whether the language is better or worse.
Why did the studies pick 2022 as the starting point?
ChatGPT became widely available in late 2022, and the researchers compare word frequencies before and after that point to separate a model effect from older trends.
Do these results apply to Canada and the United States specifically?
The samples are international. Kobak and colleagues report that estimates varied by country, but this article does not rely on a country breakdown.
Related reading
- AI in English language learning
- Practicing English speaking with AI
- AI activities for ESL students
- How GPT-3 works as a language model
More in AI Basics.
Last verified: October 2026
Sources
- Delving into LLM-assisted writing in biomedical publications through excess vocabulary, Kobak D, González-Márquez R, Horvát E-Á, Lause J, Science Advances 11(27), July 2025
- Mapping the Increasing Use of LLMs in Scientific Papers, Liang W et al., arXiv:2404.01268, 2024
- Empirical evidence of Large Language Model's influence on human spoken communication, Yakura H et al., arXiv:2409.01754, 2024, revised July 2026