A star rating or glowing testimonial for an online learning platform records how satisfied a reviewer felt, not whether students learned more. U.S. federal education law gives a sturdier yardstick: the Every Student Succeeds Act defines four levels of evidence, from a well-run experiment down to a stated rationale, and a review that cites none of them says little about effectiveness. Fake and undisclosed-insider reviews are also targeted by regulators in both the United States and Canada, so the source of a review matters as much as its score.
Key takeaways
- The Every Student Succeeds Act (ESSA) names four evidence levels: strong (experimental), moderate (quasi-experimental), promising (correlational with statistical controls), and a rationale with ongoing evaluation.
- Under section 8101(21)(B), school improvement activities funded under section 6303 can rely only on the first three levels.
- The Federal Trade Commission announced a final rule on August 14, 2024 that bans fake reviews, undisclosed insider reviews, and review suppression.
- Canada's Competition Bureau treats fake consumer reviews as misleading representations and says an undisclosed material connection, such as an employee review, is a problem even when the opinion is honest.
- Reviews of an individual product cannot be verified in general, so a buyer's best check is the study design behind the claim.
What a review of a learning platform can and cannot tell parents and teachers
Reviews answer practical questions well: whether the interface is easy to use, whether support replied, whether a child stayed interested. They are weak at the question most buyers care about, which is whether the platform changes learning outcomes. A reviewer rarely has a comparison group, a baseline score, or a way to separate the product from the teacher, the child's age, or ordinary growth over a school year. The reviewer may also be unrepresentative: people with strong feelings are the ones who write.
Those limits apply to any platform, whether it uses AI or not. No independent evaluation of a specific vendor is needed to apply them; the questions below work on any product page.
The four ESSA levels of evidence
ESSA's definition of "evidence-based" is codified in 20 U.S.C. 7801(21). It sets out three study-based levels and one rationale-based level. The statute requires the study-based levels to show a statistically significant effect on student outcomes or other relevant outcomes, from at least one well-designed and well-implemented study.
| Level | What the statute requires | What it can tell a buyer |
|---|---|---|
| 1. Strong | At least one well-designed and well-implemented experimental study | Closest to a fair test: groups were assigned by design |
| 2. Moderate | At least one well-designed and well-implemented quasi-experimental study | A comparison group exists, but it was not randomly assigned |
| 3. Promising | At least one well-designed and well-implemented correlational study with statistical controls for selection bias | A link was found; other explanations are not ruled out |
| 4. Demonstrates a rationale | A rationale based on high-quality research or positive evaluation, plus ongoing efforts to examine effects | A plausible plan, not a result |
Subparagraph (B) applies a narrower rule to interventions funded under section 6303, the school improvement provision: only levels one to three count there. ESSA governs U.S. federal programs. It is not a Canadian legal standard, though the logic of ranking designs carries over to any country.
Rules against fake and hidden-insider reviews
In the United States, the FTC's press release of August 14, 2024 announced a final rule on consumer reviews and testimonials. It bars writing or buying reviews from people who do not exist or lacked real experience, including AI-generated ones; reviews by insiders who do not disclose their connection; a company-controlled site presented as an independent review source; and using threats or false accusations to suppress negative reviews. The rule was published in the Federal Register on August 22, 2024 as 16 CFR Part 465 and lets the FTC seek civil penalties against knowing violators.
In Canada, the Competition Bureau's Deceptive Marketing Practices Digest, Volume 1 (June 10, 2015) describes "astroturfing" as creating commercial representations that masquerade as the authentic experiences of impartial consumers. It ties the practice to sections 52 and 74.01 of the Competition Act, which cover representations that are false or misleading in a material respect. The Bureau notes that an undisclosed material connection, financial or otherwise, is a problem even when the review is honest.
The FTC announcement does not make any given review false; it explains what should not appear and what a reader can look for.
A checklist for reading any education platform review
- Does the reviewer say how long the product was used, with what age group, and for which subject?
- Is any link between reviewer and seller disclosed, such as a free account, a commission, or employment?
- Does the review cite a study? If so, is the design experimental, quasi-experimental, correlational, or none of these?
- Was the study run by an independent party, or by the vendor?
- Did the study involve students similar in age, grade, and setting to the child or class in question?
- Does the review mention what the product cannot do or where it did not help?
- Are the reviews all similar in wording and posted within a short span?
FAQ
Does a high rating mean a platform improves learning?
No. A rating records satisfaction. Learning effects need a study with a comparison group, which is what ESSA's top three levels describe.
Can a platform be useful with only level four evidence?
Possibly. Level four means a plausible rationale and an evaluation planned or underway, so it is a starting point. For school improvement funds under section 6303, it is not enough.
Are paid or incentivized reviews allowed?
In the U.S., the FTC rule bars offering compensation tied to a review's sentiment, and insider reviews must disclose the connection. In Canada, the Competition Bureau treats an undisclosed material connection as a problem even when the opinion is honest.
Do these tiers apply to Canadian schools?
Not as law. ESSA covers U.S. federal education programs. Canadian parents and teachers can still use the same logic when a vendor cites a study.
Related reading
- Documented limits of AI in education
- Whether AI could replace teachers
- What elementary education covers
- Enhancing learning with AI tools
More in AI in education.
Last verified: October 2026
Sources
- Every Student Succeeds Act, 20 U.S.C. 7801(21), definition of evidence-based, Legal Information Institute, Cornell Law School
- Federal Trade Commission Announces Final Rule Banning Fake Reviews and Testimonials, FTC, August 14, 2024
- Trade Regulation Rule on the Use of Consumer Reviews and Testimonials, 16 CFR Part 465, Federal Register, August 22, 2024
- The Deceptive Marketing Practices Digest, Volume 1, Competition Bureau Canada, June 10, 2015