AI-text detectors cannot reliably show who wrote an essay. A 2023 test of 14 tools found them neither accurate nor reliable, and on January 28, 2026 a New York court annulled a university's AI-plagiarism finding that began with a Turnitin score. Schools that treat a score as proof risk punishing students who wrote their own work, and some universities have since switched the feature off.
Key takeaways
- Weber-Wulff and colleagues tested 12 public detectors and two commercial systems, including Turnitin. The tools leaned toward labeling text as human-written and did worse when text was disguised.
- Turnitin reported on May 23, 2023 a document-level false positive rate below 1 percent for documents with more than 20 percent AI writing, and it flags lower scores as less reliable.
- Mount Royal University in Calgary advised in August 2026 that instructors avoid AI detection in academic integrity cases, and Curtin University in Australia turned off Turnitin's AI detection on January 1, 2026.
- In Matter of Newby v Adelphi University, the court overturned the finding because of weak evidence and unfair process, and it did not rule on whether detectors work.
- Drafts, version history, and an oral explanation give a fairer basis for integrity decisions than a single score.
What independent testing found
The most cited academic test is Weber-Wulff and co-authors, posted on arXiv on June 21, 2023 and published that year in the International Journal for Educational Integrity (volume 19, article 26). The team examined 12 publicly available detectors and two commercial systems, Turnitin and PlagiarismCheck, asking whether they could separate human text from ChatGPT text and whether machine translation or deliberate rewording changed the result. Their conclusion was that the tools are neither accurate nor reliable, that they err mainly by calling AI text human, and that obfuscation makes them worse. The arXiv record carries the abstract. The study dates from 2023 and vendors have updated their models since, so it shows the tools as they stood then, not a ruling on current versions. No newer edition of the study turned up in searches run in October 2026.
What the vendor says about its own error rate
Turnitin's chief product officer, Annie Chechitelli, wrote on May 23, 2023 that for documents with more than 20 percent AI writing the document-level false positive rate was below 1 percent, about one human-written document in 100. The company said scores under 20 percent produced more false positives, so it began showing an asterisk on those results, and it raised the minimum length for evaluation from 150 to 300 words. These are the vendor's own figures from 2023, and they describe a defined range of scores, not every document. Even a rate that sounds small adds up across a large number of submissions.
| Human-written essays checked | Wrongly flagged at 1 percent | At 2 percent | At 5 percent |
|---|---|---|---|
| 30 (one class) | 0.3 | 0.6 | 1.5 |
| 300 (one course across a term) | 3 | 6 | 15 |
| 10,000 (one institution) | 100 | 200 | 500 |
This is an illustrative example. It assumes every essay is human-written and that the error rate is fixed, which real submissions do not guarantee, and the 2 and 5 percent columns are hypothetical rather than measured rates.
A court looks at a detector-based finding
In Matter of Newby v Adelphi University, a first-year student submitted an essay on November 7, 2024. On November 20, the instructor filed an academic integrity report that cited a Turnitin AI score of 100 percent. The student denied using generative AI and offered results from two other detectors that rated the essay human-written. A December 3 determination found him responsible, and an appeal was denied on December 7.
Justice Randy Sue Marber of the Supreme Court, Nassau County, annulled the finding on January 28, 2026 and ordered the record expunged. The court gave several reasons: an email from the instructor undermined the basis of the report, the appeal decision did not consider the student's detector results, he was not allowed an advisor of his choice as the university's own code promised, and the same person who issued the determination also decided the appeal. The opinion notes that private schools get wide discretion and that courts review only whether a school acted arbitrarily or ignored its own rules. It does not decide whether AI detectors are accurate. It is also a trial-court ruling, so it binds no other court, and the published text is marked uncorrected.
A Canadian institution steps back
In August 2026, the Generative AI in Teaching and Learning Working Group at Mount Royal University advised instructors to avoid AI detection at this time and said it was moving toward disabling Turnitin's AI writing feature. The advisory lists four reasons: both false positives and false negatives occur, detectors flag non-native English writers at high rates, a score cannot meet the university's balance-of-probabilities standard on its own, and routine checking can sour relationships between instructors and students. It quotes Turnitin as saying the tool should not be the sole basis for adverse action against a student. The advisory speaks for one university and is not a rule for other institutions.
What changed since 2023
Curtin University in Australia announced that from January 1, 2026 it would disable Turnitin's AI writing detection across all campuses and study periods, while keeping Turnitin's text-matching checks. Curtin said the aim is secure, fair assessment, and it pointed students to other approved tools. Mount Royal's August 2026 advisory points the same way in Canada. These are two institutions in two countries, not a count of how many schools have dropped detectors, and no source checked here gives such a count.
A checklist for fairer integrity processes
- Treat a detector score as a prompt for a conversation, never as a finding.
- Ask for drafts, notes, and version history, and invite the student to explain the work orally.
- State in the course outline exactly which AI uses are allowed.
- Give the student the evidence in writing and consider any evidence the student offers.
- Keep the person who hears an appeal separate from the person who made the first decision.
- Allow a student advisor, as many institutional codes already promise.
This article is general information, not legal advice. School and university discipline rules differ by institution, province, and state, and a student facing an accusation should read the applicable policy and may wish to consult an advocate or lawyer.
FAQ
Can a teacher prove AI use with a detector?
Not on a score alone. The vendor's own statements and the testing above both point to error, so an integrity finding needs other evidence such as drafts, interviews, or a pattern of work.
Did the Adelphi court say detectors are unreliable?
No. The ruling turned on the weak basis for the report and on the university's process, not on a general finding about the accuracy of detection software.
Why might detectors treat some writers unfairly?
Mount Royal University's advisory says non-native English writers are flagged at high rates, and it names other groups who may be affected.
What can a student do after a false accusation?
Collect drafts and timestamps, request the evidence in writing, use the school's appeal process, and ask about an advisor or ombudsperson.
Related reading
- guide to AI in education for policy makers
- AI in special education
- how AI technology affects schooling
- AI's harms beyond the classroom
More in AI Policy and Ethics.
Last verified: October 2026
Sources
- Testing of Detection Tools for AI-Generated Text, Weber-Wulff et al., International Journal for Educational Integrity, 2023
- AI writing detection update from Turnitin's Chief Product Officer, Turnitin, May 23, 2023
- Matter of Newby v Adelphi Univ., 2026 NY Slip Op 26021, Supreme Court, Nassau County, January 28, 2026
- Advisory: AI Writing Detection at Mount Royal University, Generative AI in Teaching and Learning Working Group, August 2026
- Curtin University to disable Turnitin AI detection tool in 2026, EdTech Innovation Hub, September 2025