No official body ranks AI systems from least to most dangerous, so a "most dangerous AI" does not exist as a documented fact. What regulators, courts, and test labs have recorded is a pattern: harm shows up where an automated match drives a consequential decision about a person and nobody checks it first, and facial recognition in security and policing is the best-documented case.
Key takeaways
- The record shows risk tied to how a system is used, not a single "most dangerous" product.
- In December 2023 the FTC announced a five-year ban on Rite Aid's use of facial recognition for surveillance after alleging thousands of false matches.
- NIST's 2019 test of 189 algorithms found false positive rates 10 to 100 times higher for some groups in one-to-one matching, varying widely by algorithm.
- Canadian commissioners found in 2021 that Clearview AI's scraping of more than 3 billion images was illegal mass surveillance.
- Safeguards that recur across cases are accuracy testing before use, human review before action, and a way to challenge a result.
Why a ranking of dangerous AI cannot be sourced
Lists of frightening AI tend to mix very different things: chatbots that make mistakes, deepfake tools, military systems, and speculative future machines. They cannot be compared on one scale, because harm depends on who relies on the output, how often it is wrong, and what happens after an error. A weak model that drafts a birthday message is low risk. The same error rate in a system that decides who is stopped at a store door, arrested, or denied a benefit is a different matter.
That is why the documented cases below are about deployment, not about a brand. They involve systems that were in use, with named regulators or courts that reviewed what happened. Speculative scenarios are left to other articles on this site.
Case one: a retailer's watchlist
On December 19, 2023, the FTC announced a settlement with the pharmacy chain Rite Aid. According to the agency, the company used AI-based facial recognition from 2012 to 2020 in hundreds of stores to flag customers it suspected of shoplifting. The FTC alleged that the system produced thousands of false-positive matches, that employees followed, searched, and accused customers on the strength of those matches, and that women and people of color were disproportionately affected.
The complaint listed process failures rather than a single technical flaw: no assessment of misidentification risk, no accuracy testing before rollout, low-quality images accepted, no tracking of false positives afterward, and little training for staff. The order bans the use of facial recognition for surveillance for five years and requires deletion of the images and of algorithms built from them. Judge Kelley B. Hodge of the Eastern District of Pennsylvania entered the stipulated order on February 26, 2024. The FTC's press release on the Rite Aid order gives the allegations and terms.
Case two: an arrest built on a match
In January 2020, Detroit police arrested Robert Williams outside his home. According to the American Civil Liberties Union, which represented him, officers had sent a blurry surveillance still from a 2018 store theft for a facial recognition search, received a possible match to his expired driver's license photo, and then showed his picture in a lineup to a loss-prevention contractor who had not been at the store that day. He spent about thirty hours in detention.
The lawsuit was settled on June 28, 2024 in the federal court for the Eastern District of Michigan. Under the terms described by the ACLU, Detroit police must back up a facial recognition result with independent, reliable evidence before any arrest, officers must be trained on the technology and its dangers, and past cases since 2017 in which it was used to obtain an arrest warrant are to be audited. The settlement does not ban the technology. It adds a human evidence requirement.
What test data says about error rates
These failures raise a technical question: do the algorithms err evenly across people? NIST's 2019 demographic study evaluated 189 algorithms from 99 developers using 18.27 million images of 8.49 million people drawn from government databases. In one-to-one matching, such as confirming that two photos show the same person, many algorithms had false positive rates for Asian and African American faces that were 10 to 100 times higher than for Caucasian faces. In one-to-many searches against a database of 1.6 million domestic mugshots, false positives were higher for African American women.
NIST also stressed the limits. Algorithms performed differently, some Asian-developed algorithms did not show the same gap in one-to-one matching, and the most equitable algorithms were among the most accurate. The study tested algorithms, not finished commercial products, and did not examine causes. A blanket claim that all facial recognition is biased, or that none is, does not fit the data.
Case three: a database built without consent
A different kind of danger came from how images were collected. On February 3, 2021, the federal privacy commissioner of Canada, together with the commissioners of Quebec, British Columbia, and Alberta, reported that Clearview AI had violated federal and provincial privacy laws by scraping more than 3 billion images from across the internet to build a facial recognition service. The commissioners called this mass surveillance and said the biometric information was collected without knowledge or consent. Clearview disagreed with the findings, and the authorities recommended that it stop serving Canadian clients and delete images of people in Canada.
On June 10, 2021, the federal commissioner found that the Royal Canadian Mounted Police had violated the Privacy Act by using Clearview's service, running hundreds of searches against a database compiled unlawfully. The RCMP disagreed with the finding but agreed to implement the commissioner's recommendations on policies, systems, and training.
Safeguards that recur in the record
| Case | What the system did | Missing safeguard on record | Result |
|---|---|---|---|
| Rite Aid | Flagged shoppers as suspects | No pre-deployment accuracy testing, no monitoring of false positives, little staff training | Five-year surveillance ban, deletion of images and models |
| Detroit arrest | Produced a lead from a low-quality image | No independent evidence before arrest | Settlement requiring corroboration, training, and an audit |
| Clearview AI | Matched faces against scraped photos | No consent, no lawful purpose | Findings of illegality in Canada, police use ruled a violation |
The pattern is consistent: test before use, measure errors after launch, keep a person accountable for the final decision, and give people a way to contest the outcome. Each safeguard is cheap compared with a wrongful accusation.
This article gives general information drawn from regulatory and court records. It is not legal advice, and the descriptions of allegations and findings do not establish facts beyond what those bodies reported. Readers with a specific legal concern should consult a qualified professional or the primary documents listed below.
FAQ
Which AI system has caused the most harm?
No source ranks systems by total harm. Records of incidents, enforcement actions, and court cases are incomplete and cover different kinds of damage, so any single "worst" label would be a guess.
Is facial recognition banned in the United States or Canada?
Not nationwide in either country. The Rite Aid order is a five-year ban for one company, the Detroit settlement limits how one police department acts on results, and Canadian commissioners have found specific uses unlawful under existing privacy laws.
Does facial recognition work worse for some groups?
For many algorithms NIST measured higher false positive rates for some groups, but the size of the gap differed by algorithm and by task. NIST also noted that the most equitable algorithms were among the most accurate.
Can a match from AI alone justify an arrest?
In Detroit, no. Under the June 2024 settlement, police need independent, reliable evidence linking a suspect to a crime. Rules in other places differ and should be checked locally.
Related reading
- Dangers and risks posed by AI
- Documented benefits of AI
- How organizations use AI
- The AI sentience question
More in AI Policy and Ethics.
Last verified: October 2026
Sources
- Rite Aid Banned from Using AI Facial Recognition After FTC Says Retailer Deployed Technology without Reasonable Safeguards, Federal Trade Commission, December 19, 2023
- Commission order reopening proceedings and modifying the 2010 order, Rite Aid Corporation, Federal Trade Commission, 2024
- Williams v. City of Detroit, case page, American Civil Liberties Union, 2024
- NIST Study Evaluates Effects of Race, Age, Sex on Face Recognition Software (NISTIR 8280), National Institute of Standards and Technology, December 19, 2019
- Clearview AI's unlawful practices represented mass surveillance, joint news release, Office of the Privacy Commissioner of Canada, February 3, 2021
- RCMP's use of Clearview AI violated the Privacy Act, Office of the Privacy Commissioner of Canada, June 10, 2021