Controlled studies show that AI assistance can make short professional writing faster and better rated: in a 2023 Science experiment with 453 college-educated professionals, ChatGPT cut average task time by 40 percent and raised rated quality by 18 percent. Evidence on what students lose or keep is thinner, and the main study that suggests lower engagement is a small preprint. Two rules are settled in scholarly publishing: a chatbot cannot be an author, and its use must be disclosed.
Key takeaways
- The best-known randomized result covers professionals writing workplace documents, not students writing essays.
- An EEG study of 54 participants found weaker brain connectivity in the group that wrote with a chatbot, but it is a preprint with a small sample.
- Seven AI-text detectors flagged over half of a set of essays by non-native English writers as AI-generated, according to a Stanford-led study.
- The ICMJE recommendations say chatbots should not be listed as authors and that authors must disclose AI use and answer for the result.
- Course rules vary, so a student's first source of truth is the syllabus or the instructor.
What the evidence covers and what it leaves out
Most claims about AI and writing skill rest on a handful of studies, and each answers a narrow question. The table sets out what each one tested and where it stops.
| Study | Design | Finding | Limit |
|---|---|---|---|
| Noy and Zhang, Science, July 2023 | Preregistered online experiment, 453 college-educated professionals, half given ChatGPT | Average time fell 40 percent; rated quality rose 18 percent; gaps between workers narrowed | Mid-level professional writing tasks, not coursework, and no measure of learning |
| Kosmyna et al., arXiv, June 2025 (revised December 31, 2025) | 54 participants wrote essays with a chatbot, a search engine, or no tool; EEG recorded | Connectivity was weakest in the chatbot group; those users reported the least ownership of their essays | Small sample, only 18 people in the final session, and the arXiv listing does not state peer review |
| Liang et al., 2023 | Seven AI-text detectors tested on 91 TOEFL essays and 88 U.S. eighth-grade essays | Detectors were near-perfect on the U.S. essays but flagged over half of the TOEFL essays | Detectors tested in 2023, and newer versions were not evaluated |
None of the three measures whether a student who drafts with AI writes better a year later. That gap matters more than any single number.
Speed and quality gains in the Science experiment
Shakked Noy and Whitney Zhang randomly assigned occupation-specific writing tasks to 453 college-educated professionals and gave half of them ChatGPT. The group with the tool finished faster, by 40 percent on average, and independent evaluators rated the output 18 percent higher. The gap between workers narrowed. People who had access were also more likely to use ChatGPT in their jobs afterward.
The setting was a workplace task with a clear deliverable. An academic essay serves another purpose: the writing is a way of thinking, and the grade measures the thinking. A result about output cannot be read as a result about learning.
A cautionary preprint, read carefully
The study often summarized as "Your Brain on ChatGPT" asked 54 people to write essays over three sessions in three conditions: with a chatbot, with a search engine, or with no tool. EEG recordings showed the strongest connectivity in the no-tool group and the weakest in the chatbot group. In a fourth session, 18 participants swapped conditions, and those who moved from the chatbot to unaided writing showed reduced alpha and beta connectivity, which the authors read as under-engagement.
The authors call the pattern cognitive debt. The data show an association in a small group, not a lasting effect on skill, and the paper is listed on arXiv without a stated peer review. It is a reason to run larger studies, not a verdict.
Why AI detectors are a poor referee
Liang and colleagues at Stanford tested seven widely used detectors on writing samples. On U.S. eighth-grade essays the detectors were close to accurate. On TOEFL essays written by non-native speakers, they misclassified over half as AI-generated, with an average false positive rate of 61.22 percent in the preprint. All seven flagged 18 of the 91 TOEFL essays, and at least one flagged 89 of them. The essays flagged by every detector had significantly lower perplexity, a measure of how predictable the wording is, and the authors suggest that detectors penalize writers with limited linguistic range. They caution against using such tools in evaluative or educational settings.
For a student, the practical point is that a flag is not proof. For an instructor, a detector score alone is a weak basis for an accusation, and drafts, version history, and a conversation about the work carry more weight.
Disclosure rules in scholarly publishing
The International Committee of Medical Journal Editors (ICMJE) gives the clearest published rule for authors. Chatbots such as ChatGPT should not be listed as authors, because they cannot take responsibility for the work. Authors should disclose at submission whether AI-assisted tools were used and explain how, in the cover letter and, where relevant, in the manuscript. Humans remain accountable for everything submitted, including checking AI output for inaccuracy, bias, and plagiarism, and AI-generated content cannot be cited as a primary source. Failing to disclose may lead to corrective action and, in some cases, be treated as misconduct.
These rules govern medical journals that follow ICMJE. Universities, courses, and other publishers set their own, and many differ from instructor to instructor.
A checklist for writers who use AI
- Read the syllabus or ask the instructor which uses are allowed: brainstorming, outlining, grammar edits, or drafting.
- Keep dated drafts and notes so the writing process can be shown.
- Verify every fact, quotation, and reference an AI tool supplies against the original source.
- Write the thesis and the argument unaided, then compare with any AI suggestion.
- Record which tool and which prompts were used, and disclose them where a rule requires it.
- Do not submit text that cannot be defended in a conversation with the instructor.
FAQ
Does using AI make students worse writers?
No study reviewed here shows that. One small preprint found lower brain connectivity and weaker ownership of essays in a chatbot group, and the randomized workplace study found faster, higher-rated output. Long-term effects on student skill remain unmeasured.
Can an AI detector prove that a student cheated?
No. In the Stanford-led test, detectors wrongly flagged a majority of non-native English essays. A detector result can start a conversation, but it should not decide a case.
Can a chatbot be listed as an author?
Under the ICMJE recommendations, no. Authorship carries responsibility for accuracy and integrity, which a chatbot cannot take, so the human authors must disclose its use and stand behind the text.
Where can the rules for a specific course be found?
In the syllabus, the course learning platform, or the institution's academic integrity policy. If the rule is unclear, asking the instructor before submitting is the safest route.
Related reading
- AI and academic integrity
- AI activities for ESL students
- Using AI to write letters
- AI support for students with disabilities
More in AI in education.
Last verified: October 2026
Sources
- Recommendations on AI use by authors, International Committee of Medical Journal Editors, 2026
- Liang W, Yuksekgonul M, Mao Y, Wu E, Zou J, GPT detectors are biased against non-native English writers, Patterns, 2023 (arXiv version 3, July 10, 2023)
- Noy S, Zhang W, Experimental evidence on the productivity effects of generative artificial intelligence, Science, July 2023
- Kosmyna N et al., Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task, arXiv:2506.08872, 2025