Evidence that AI changes higher education is narrow so far: one randomized trial shows a course chatbot raised the share of students earning an A or B by four percentage points, and one campus survey shows generative AI use above 80 percent at a selective college. Neither study shows broad change in degrees earned or graduate outcomes. The honest reading is that AI has measurably shifted specific tasks and habits, while the question of transformation remains open.
Key takeaways
- A pre-registered trial at Georgia State University randomized 2,483 students in two online courses and found a text-message chatbot raised the chance of an A or B by four percentage points.
- That chatbot was non-generative, drew on a pre-built knowledge base, and left course staff to answer questions it could not.
- The effects did not carry over to students' other courses or to the next semester.
- At Middlebury College, a survey found 82.5 percent of respondents used generative AI for academic purposes, and uses that help the student learn were more common than uses that do the work for the student.
The Georgia State trial: reminders, not answers
The most rigorous higher education evidence found for this article is a working paper by Katharine Meyer, Lindsay Page, and colleagues, posted in its current version in June 2026. Georgia State University randomized students in two large online courses, Introduction to American Government and Principles of Microeconomics. Across three terms from fall 2021 through fall 2022, 1,568 Government students were randomized, and 915 Microeconomics students were randomized in 2022-23. Half received two to three scheduled text messages per week, about 40 over the semester, with due-date reminders and nudges for missing work. Students could text the bot at any hour.
The tool was not a generative chatbot. It answered from a prepared knowledge base built with the instructors, and messages it could not match were flagged for the teaching assistant. The paper reports that the likelihood of an A or B rose by four percentage points across the two courses. In Microeconomics, women assigned to the chatbot earned final grades seven points higher than women in the control group, were 11 percentage points more likely to earn an A or B, and were 10 points less likely to receive a D, an F, or a withdrawal. There was no such effect for men.
What the trial did not show
The chatbot changed behavior in the course where it ran. Treated students were more likely to attend university tutoring, and there is suggestive evidence of more homework completion in Microeconomics. The authors found no spillover to students' grades in other courses that term and no effect on later outcomes the next semester. Their interpretation is that students followed the specific advice, such as attending tutoring, without carrying a general study skill into other settings.
Two further cautions apply. The paper is a working paper, and its listing does not mention journal publication. The chatbot was built in a partnership between Georgia State, outside researchers, and a technology company, a relationship the paper itself discloses.
The Middlebury survey: wide use, uneven by field
Economists Zara Contractor and Germán Reyes surveyed students at Middlebury College, a selective liberal arts college in Vermont with about 2,800 undergraduates, between December 2024 and February 2025. They received 739 responses, a 26.8 percent response rate, and analyzed 634. Among those students, 82.5 percent reported using generative AI for academic purposes, up from less than 10 percent before spring 2023. Adoption ranged from over 90 percent in natural sciences majors to under 50 percent in literature.
The authors separated uses that augment learning, such as getting explanations and feedback, from uses that automate coursework. Among AI users, the average use rate across augmentation tasks was 56.0 percent, against 45.4 percent across automation tasks. Their conclusion is that policies should distinguish between uses that enhance learning and uses that substitute for it. The sample comes from one wealthy, selective college, so it cannot stand in for community colleges, large public universities, or Canadian institutions.
Two studies side by side
| Item | Georgia State chatbot trial | Middlebury survey |
|---|---|---|
| Design | Pre-registered randomized trial | Survey with post-stratification weights |
| Participants | 1,568 and 915 students in two courses | 634 analyzed students, one college |
| Type of AI | Non-generative text-message bot | Generative AI tools chosen by students |
| Main result | Four-point rise in A or B probability | 82.5 percent used it for academic purposes |
| Main limit | One university, no later effect | Self-report, one selective college |
The Canadian policy picture
Canadian research universities have responded with guidance more than with measured outcomes. In September 2024, U15 Canada, the association of research-intensive universities, published "Navigating AI in Teaching and Learning: Values, Principles and Leading Practices." It commits members to an ethical framework and names bias from training data, accuracy, transparency, privacy, and copyright among the challenges. Guidance of this kind states intentions. It does not measure whether students learn more or less.
How to read a claim that AI is changing universities
- Which task changed: reminders, tutoring, grading, writing, or admissions?
- Was there a control group, or only before-and-after talk?
- Does the result persist after the term, or only inside the course?
- Who ran the study, and who built the tool?
- Is the sample a selective college, a large public system, or a community college?
FAQ
Does AI improve grades in college courses?
In one randomized trial, a non-generative course chatbot raised the likelihood of an A or B by four percentage points. No comparable randomized evidence was gathered here for generative tools used freely by students, and that trial's effects did not extend to other courses.
How many college students use generative AI?
At Middlebury College, 82.5 percent of analyzed survey respondents did, in a 2024-25 survey. That is a single selective college, so the figure should not be read as a national rate.
Is the Georgia State chatbot the same as ChatGPT?
No. The authors describe it as non-generative AI that answers from a knowledge base prepared with the course staff and refers unmatched questions to a teaching assistant.
Has AI changed who earns a degree?
The studies reviewed do not measure degree completion. The Georgia State authors found no effect on the next semester, so any link to graduation remains unproven.
Related reading
- Whether AI could replace teachers
- AI and the learning experience
- A decade of AI in special education
- Effects of AI technology in schools
More in AI in education.
Last verified: October 2026
Sources
- Let's Chat: Leveraging Chatbot Outreach for Improved Course Performance, Meyer K, Page L, et al., EdWorkingPaper 22-564, Annenberg Institute at Brown University, June 2026
- Generative AI in Higher Education: Evidence from an Elite College, Contractor Z, Reyes G, arXiv 2508.00717, version 2, April 2026 (source page)
- Navigating AI in Teaching and Learning: Values, Principles and Leading Practices, U15 Canada, September 2024