Classroom AI Prompts: What Trials Say About Prompt Libraries

What a Harvard physics trial and a 900-tutor study show about AI prompts in teaching, with a template for tested, dated classroom prompt entries.

Last verified: October 20266 min read

A classroom prompt library is a shared set of tested instructions for an AI model, and the published evidence says the instruction text is only part of what makes it work. In a 2025 randomized trial at Harvard, a tutor driven by a written system prompt beat an in-class active learning session, yet its authors report that the prompt alone could not reliably guide multi-part problems. Design around the prompt, such as step-by-step structure and checked answers, did much of the work.

Key takeaways

  • A prompt library is worth keeping only if each entry has been tested and checked against correct answers.
  • In the Harvard physics trial (194 students analyzed), the AI tutor's median gain was more than double the in-class group's, but the tutor relied on pre-written solutions and structured steps.
  • A 2024 trial with 900 tutors found that AI suggestions raised student mastery by four percentage points, and by nine for students of lower-rated tutors.
  • Federal guidance in Canada tells public servants to give context, audience, length, and tone, to refine with follow-up prompts, and never to enter personal information into public tools.
  • Prompts for classroom use should not contain student names, grades, or other identifying details.

What a prompt library can and cannot hold

Collections of ready-made prompts for teaching are easy to find, but most are untested lists. A prompt is an instruction written in ordinary language, and the same wording can give different answers from one run to the next or from one model version to another. A library therefore has two jobs: store wording that has worked, and record what it was tested on, with which model, and on what date.

The evidence below shows why the second job matters. The strongest studies of AI in teaching did not test a single clever sentence. They tested a system with a prompt, course content, and human checks around it.

What the Harvard physics trial did with its prompts

Greg Kestin, Kelly Miller, and colleagues ran a randomized crossover trial in a Harvard introductory physics course for life sciences students in fall 2023. Of 233 enrolled students, 194 were eligible for analysis. Each student studied one lesson, on surface tension or fluid flow, with an AI tutor at home and the other lesson in a class using active learning, with assignment randomized by small peer-instruction groups. Median post-test scores were 4.5 for the AI group and 3.5 for the in-class group, and the median gain was more than double. The linear regression effect size was 0.63, which the authors call an underestimate because of a ceiling effect.

The paper in Scientific Reports describes how the tutor was built. A system prompt set rules for active engagement, managing cognitive load, and encouraging a growth mindset. The authors found the prompt alone could not reliably scaffold multi-part problems, so the platform walked students through each part in order. The prompts also carried step-by-step solutions written in advance by instructors, which kept the model accurate. The underlying model was GPT-4, and development took several months. The authors do not claim the result holds for every course. The study covered one course at one institution, targeted understanding and application rather than higher-order synthesis, and the tests did not count toward grades.

A second trial: prompts that help human tutors

Rose Wang, Susanna Loeb, Dora Demszky, and colleagues tested a different use. In their preregistered randomized trial, 900 tutors working with 1,800 K-12 students from historically under-served communities received AI-generated suggestions during live sessions, a system called Tutor CoPilot. Students of tutors with the tool were four percentage points more likely to master topics, and students of lower-rated tutors gained nine points. The analysis of more than 550,000 messages found that tutors used more guiding questions and gave away answers less often. The authors report a cost of about USD 20 per tutor per year, and tutor interviews flagged some suggestions that did not fit the students' grade level. Details are in the arXiv paper.

The lesson for a library is that a prompt can ask for hints and questions instead of answers, and that a teacher still has to judge whether the output fits the student in front of them.

Anatomy of a prompt entry

The table below is an illustrative template for one library entry. The fields follow the general advice in the Government of Canada's guide on generative AI, which asks users to be clear and specific, supply context, identify the audience, and state length and tone. The example column is invented for illustration and is not drawn from either trial.

FieldWhat it recordsIllustrative example
PurposeThe teaching task in one lineDraft three practice questions on fractions
Context and audienceGrade, prior knowledge, language levelGrade 5 class, finished adding fractions with like denominators
ConstraintsLength, tone, what to avoidShort sentences, no names of real students, no trick wording
Output formatShape of the answerNumbered questions, answer key listed separately
Tested onModel, date, number of trial runsModel name, October 2026, five runs
Known failuresWhere wrong or off-level output appearedTwo answer-key errors in five runs
ReviewerWho checked it against the curriculumGrade team lead

Handling student information

The federal guide states that public servants must not input personal information into publicly available online generative AI tools, and it says generated content should not be treated as authoritative. Those rules are written for government staff, not teachers, and school boards and districts set their own policies. The same reasoning is a sensible default for any shared library: write prompts with invented or generic examples, keep names and grades out of them, and check any answer key before it reaches a class.

A short test routine before a prompt enters the library

  • Run the prompt at least five times and compare the outputs.
  • Check every factual statement and every worked answer against the textbook or curriculum.
  • Read the output against the target grade level and revise it if the vocabulary runs above it.
  • Record the model and date, and retest after a model update.
  • Remove any entry that cannot be checked quickly.

FAQ

Is a bigger prompt library better?

No evidence supports that. The trials above tested designed systems, not large collections. A small set of entries that have been run, checked, and dated is more useful than hundreds of untested ones.

Can a prompt alone turn a chatbot into a good tutor?

The Harvard authors report that their system prompt could not reliably guide multi-part problems by itself. Their tutor also used instructor-written solutions and a step-by-step structure, so the result should not be credited to prompt wording alone.

Should prompts ask the AI for answers or for hints?

The Tutor CoPilot analysis found that tutors using the tool offered more guiding questions and gave away answers less often. Whether hints or answers suit a given lesson is a teaching decision, and hint-style prompts need testing like any other.

Do these results apply to Canadian classrooms?

Both trials ran in the United States. The Harvard study used university students and the tutoring study used K-12 students in tutoring programs. Neither tested a Canadian school system, so local pilots and board policy still apply.

Last verified: October 2026

Sources

  • AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting, Kestin G, Miller K, Klales A, Milbourne T, Ponti G, Scientific Reports 15:17458, June 3, 2025
  • Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise, Wang R, Ribeiro A, Robinson C, Loeb S, Demszky D, arXiv:2410.03017, 2024, revised 2026
  • Guide on the use of generative artificial intelligence, Government of Canada, page details dated September 24, 2026

New to artificial intelligence?

Start with the basics, then follow the guides to tools, rules, and effects on society.

Start with AI Basics