A classroom prompt library is a shared set of tested instructions for an AI model, and the published evidence says the instruction text is only part of what makes it work. In a 2025 randomized trial at Harvard, a tutor driven by a written system prompt beat an in-class active learning session, yet its authors report that the prompt alone could not reliably guide multi-part problems. Design around the prompt, such as step-by-step structure and checked answers, did much of the work.
Key takeaways
- A prompt library is worth keeping only if each entry has been tested and checked against correct answers.
- In the Harvard physics trial (194 students analyzed), the AI tutor's median gain was more than double the in-class group's, but the tutor relied on pre-written solutions and structured steps.
- A 2024 trial with 900 tutors found that AI suggestions raised student mastery by four percentage points, and by nine for students of lower-rated tutors.
- Federal guidance in Canada tells public servants to give context, audience, length, and tone, to refine with follow-up prompts, and never to enter personal information into public tools.
- Prompts for classroom use should not contain student names, grades, or other identifying details.
What a prompt library can and cannot hold
Collections of ready-made prompts for teaching are easy to find, but most are untested lists. A prompt is an instruction written in ordinary language, and the same wording can give different answers from one run to the next or from one model version to another. A library therefore has two jobs: store wording that has worked, and record what it was tested on, with which model, and on what date.
The evidence below shows why the second job matters. The strongest studies of AI in teaching did not test a single clever sentence. They tested a system with a prompt, course content, and human checks around it.
What the Harvard physics trial did with its prompts
Greg Kestin, Kelly Miller, and colleagues ran a randomized crossover trial in a Harvard introductory physics course for life sciences students in fall 2023. Of 233 enrolled students, 194 were eligible for analysis. Each student studied one lesson, on surface tension or fluid flow, with an AI tutor at home and the other lesson in a class using active learning, with assignment randomized by small peer-instruction groups. Median post-test scores were 4.5 for the AI group and 3.5 for the in-class group, and the median gain was more than double. The linear regression effect size was 0.63, which the authors call an underestimate because of a ceiling effect.
The paper in Scientific Reports describes how the tutor was built. A system prompt set rules for active engagement, managing cognitive load, and encouraging a growth mindset. The authors found the prompt alone could not reliably scaffold multi-part problems, so the platform walked students through each part in order. The prompts also carried step-by-step solutions written in advance by instructors, which kept the model accurate. The underlying model was GPT-4, and development took several months. The authors do not claim the result holds for every course. The study covered one course at one institution, targeted understanding and application rather than higher-order synthesis, and the tests did not count toward grades.
A second trial: prompts that help human tutors
Rose Wang, Susanna Loeb, Dora Demszky, and colleagues tested a different use. In their preregistered randomized trial, 900 tutors working with 1,800 K-12 students from historically under-served communities received AI-generated suggestions during live sessions, a system called Tutor CoPilot. Students of tutors with the tool were four percentage points more likely to master topics, and students of lower-rated tutors gained nine points. The analysis of more than 550,000 messages found that tutors used more guiding questions and gave away answers less often. The authors report a cost of about USD 20 per tutor per year, and tutor interviews flagged some suggestions that did not fit the students' grade level. Details are in the arXiv paper.
The lesson for a library is that a prompt can ask for hints and questions instead of answers, and that a teacher still has to judge whether the output fits the student in front of them.
Anatomy of a prompt entry
The table below is an illustrative template for one library entry. The fields follow the general advice in the Government of Canada's guide on generative AI, which asks users to be clear and specific, supply context, identify the audience, and state length and tone. The example column is invented for illustration and is not drawn from either trial.
| Field | What it records | Illustrative example |
|---|---|---|
| Purpose | The teaching task in one line | Draft three practice questions on fractions |
| Context and audience | Grade, prior knowledge, language level | Grade 5 class, finished adding fractions with like denominators |
| Constraints | Length, tone, what to avoid | Short sentences, no names of real students, no trick wording |
| Output format | Shape of the answer | Numbered questions, answer key listed separately |
| Tested on | Model, date, number of trial runs | Model name, October 2026, five runs |
| Known failures | Where wrong or off-level output appeared | Two answer-key errors in five runs |
| Reviewer | Who checked it against the curriculum | Grade team lead |
Handling student information
The federal guide states that public servants must not input personal information into publicly available online generative AI tools, and it says generated content should not be treated as authoritative. Those rules are written for government staff, not teachers, and school boards and districts set their own policies. The same reasoning is a sensible default for any shared library: write prompts with invented or generic examples, keep names and grades out of them, and check any answer key before it reaches a class.
A short test routine before a prompt enters the library
- Run the prompt at least five times and compare the outputs.
- Check every factual statement and every worked answer against the textbook or curriculum.
- Read the output against the target grade level and revise it if the vocabulary runs above it.
- Record the model and date, and retest after a model update.
- Remove any entry that cannot be checked quickly.
FAQ
Is a bigger prompt library better?
No evidence supports that. The trials above tested designed systems, not large collections. A small set of entries that have been run, checked, and dated is more useful than hundreds of untested ones.
Can a prompt alone turn a chatbot into a good tutor?
The Harvard authors report that their system prompt could not reliably guide multi-part problems by itself. Their tutor also used instructor-written solutions and a step-by-step structure, so the result should not be credited to prompt wording alone.
Should prompts ask the AI for answers or for hints?
The Tutor CoPilot analysis found that tutors using the tool offered more guiding questions and gave away answers less often. Whether hints or answers suit a given lesson is a teaching decision, and hint-style prompts need testing like any other.
Do these results apply to Canadian classrooms?
Both trials ran in the United States. The Harvard study used university students and the tutoring study used K-12 students in tutoring programs. Neither tested a Canadian school system, so local pilots and board policy still apply.
Related reading
- Review of current trends in AI in education
- How AI training is implemented
- Whether AI learns and adapts to new information
- Education code and the school system
More in AI in education.
Last verified: October 2026
Sources
- AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting, Kestin G, Miller K, Klales A, Milbourne T, Ponti G, Scientific Reports 15:17458, June 3, 2025
- Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise, Wang R, Ribeiro A, Robinson C, Loeb S, Demszky D, arXiv:2410.03017, 2024, revised 2026
- Guide on the use of generative artificial intelligence, Government of Canada, page details dated September 24, 2026