Education support covers the services that keep a student moving through school or college: reminders about deadlines, help with registration and aid forms, and quick answers to routine questions. Two randomized studies at Georgia State University tested a text-message chatbot for this work. A 2017 trial found incoming students were 3.3 percentage points more likely to enroll on time, and a 2024 working paper found students in two large online courses were four percentage points more likely to earn an A or B.
Key takeaways
- The 2024 course chatbot was non-generative: it answered from a prepared knowledge base, and a teaching assistant reviewed questions it could not answer.
- The 2017 trial measured on-time enrollment among admitted students. The 2024 working paper measured course grades among 2,483 students in two courses.
- Effects were modest: about four percentage points on the chance of an A or B, against a control rate of 61 percent.
- The authors say the rollout needed a semester of piloting and continuing human monitoring.
- Results come from one university and cannot be assumed for K-12 schools or for generative chatbots.
Two kinds of support a chatbot can offer
Student support falls into two groups. Administrative support covers tasks such as confirming a place, resolving a registration hold, or refiling a federal aid application. Academic support covers course work, such as knowing when an assignment is due and where to get tutoring. The Georgia State research team describes the first as a long-running line of text-message outreach and the second as a newer test of whether the same approach can change grades.
The distinction matters because the evidence differs. Administrative nudges have a clear endpoint, since a student either enrolled or did not. Academic nudges must show that messages changed study behavior and, through it, results.
Trial one: keeping admitted students on track for fall
Lindsay Page and Hunter Gehlbach published the first study in AERA Open in 2017. Their field experiment at Georgia State used conversational AI to send personalized text messages to thousands of would-be freshmen during the summer between high school and college, when college-related tasks can keep admitted students from arriving in the fall. Students who had committed to the university and were assigned to the system did better on pre-enrollment requirements and were 3.3 percentage points more likely to enroll on time. The authors describe the effect as comparable to earlier interventions, with much less burden on university staff.
Trial two: a chatbot inside two online courses
A team led by Katharine Meyer and Lindsay Page tested a course-specific version in a working paper dated November 14, 2024. In fall 2021, spring 2022, and fall 2022 they randomized 1,568 students in an introductory American government course, and in the 2022-23 academic year they randomized 915 students in sections of microeconomics. Half of each group received two to three scheduled texts a week covering due dates, nudges about missing work, and an invitation to ask questions. Students could text the bot at any hour and receive answers drawn from a prepared knowledge base. Questions it could not answer were flagged for the course teaching assistant.
| Measure | Reported result |
|---|---|
| Earning an A or B, both courses pooled | Four percentage points higher than a control rate of 61 percent |
| Earning an A, both courses pooled | Four percentage points higher than a control rate of 36 percent |
| Earning a D, F, or withdrawing, government course | Three percentage points lower than a control rate of 18 percent |
| Dropping the course, microeconomics | Three percentage points lower than a control rate of eight percent |
| Women in microeconomics | Final grades seven points higher; no effect found for men |
| Students who opted out of texts | Four percent in government, three percent in microeconomics |
The paper reports that the pooled sample showed no significant change in the rate of D, F, or withdrawal. It calls the evidence on homework completion and tutoring attendance suggestive, and it found no consistent change in student attitudes on the survey items it measured. The survey ran only in the government course and about 68 percent of students completed it. Among respondents, 82 percent recommended continuing the chatbot and expanding it to other courses.
What stood behind the bot
The authors state that setting up the system took substantial upfront investment from the instructional team and the university support office. A semester of piloting let staff write messages that matched the syllabus and train the bot on likely questions, and the paper says running it well requires ongoing human monitoring. In one example, a student reported being dropped for unpaid tuition. The bot replied with a contact for student financial services, and the teaching assistant followed up so the student could still reach the textbook during the hold.
That arrangement is the reason the results should not be read as software replacing support staff. The chatbot handled routine messages and routing while people handled the exceptions.
Privacy and consent in the setup
All students in the course analysis had opted in to text messages, and the paper says about 86 percent of incoming students at Georgia State each term opt in. In the United States, the Family Educational Rights and Privacy Act regulations define an eligible student as one who has reached 18 years of age or is attending an institution of postsecondary education, which is the category that applies when a university texts students about their own course performance. Whether a given messaging program meets those requirements is a question for the institution's counsel. This article gives general information, not legal advice, and it does not endorse any product.
Questions to ask before relying on a support chatbot
- Is the outcome administrative, such as enrollment, or academic, such as grades, and has it been tested in that form?
- Does the tool answer from a vetted knowledge base, or does it generate free text?
- Who reads the questions it cannot answer, and how fast do they reply?
- Can students opt out, and what share do?
- Were results measured against a randomly assigned comparison group?
- Do the students and courses in the study resemble the ones where the tool would be used?
FAQ
Did the Georgia State chatbot use generative AI?
No. The 2024 paper describes it as non-generative: it answered from a prepared knowledge base and passed other questions to a teaching assistant.
How large were the grade effects?
About four percentage points on the chance of an A or B. The control group rate was 61 percent, so the paper describes this as roughly a seven percent increase.
Has the 2024 paper been peer reviewed?
The version checked is an EdWorkingPaper dated November 14, 2024, which is a working paper. A journal version was not located, so the figures here should be treated as preliminary.
Do the results apply to high schools or Canadian institutions?
The studies took place at one large public university in the United States. No comparable Canadian trial was verified for this article, so the results describe that setting rather than schools in general.
Related reading
- OpenAI products for students and teachers
- elementary education explained
- what education codes require
- how education regulation works
More in AI in education.
Last verified: October 2026
Sources
- Page, L. C., and Gehlbach, H., an artificially intelligent virtual assistant for the transition to college (field experiment at Georgia State), AERA Open, 2017
- Meyer, K., Page, L. C., et al., Let's Chat: Leveraging Chatbot Outreach for Improved Course Performance, EdWorkingPaper No. 22-564, Annenberg Institute at Brown University, November 2024
- 34 CFR 99.3, definitions under the Family Educational Rights and Privacy Act regulations, U.S. Code of Federal Regulations