People are poor at telling AI-generated images from human-made art, and software detectors do better but can be fooled. In a 2024 study presented at the ACM Conference on Computer and Communications Security, general online participants labeled images correctly 59.23 percent of the time, professional artists 75.32 percent, and the commercial detector Hive 98.03 percent on unaltered images. A team of one expert artist plus Hive did best of all, so no single check was enough.
Key takeaways
- A chance-level guess scores 50 percent; general participants in the University of Chicago study scored 59.23 percent.
- Thirteen expert artists reached 83.00 percent, but more than one in five human-made images was wrongly flagged as AI.
- Hive never flagged a human artwork in the test, yet its score on AI images fell sharply once Glaze, a style-protection filter, was applied.
- The study's authors include the creators of Glaze, a fact worth weighing when reading the results.
- As of October 2026 the paper remains the reference for these figures: its detector results describe 2024 products and generators, and no comparable follow-up was located.
What the study tested
The experiment, a paper titled "Organic or Diffused" on distinguishing human art from AI-generated images, appeared in the proceedings of ACM CCS 2024, held October 14 to 18, 2024, in Salt Lake City. The authors, from the University of Chicago, assembled 280 human artworks across seven styles and 350 images from five image generators. They added 40 hybrid images, which mix human and AI work, and 70 upscaled human pieces. Eight judges then sorted the material: five automated detectors and three human groups. The groups were about 180 paid online participants, 3,803 professional artists recruited through a survey, and 13 expert artists with experience spotting AI images. The paper is posted on arXiv, and the figures below come from it.
How well humans and software did
Accuracy is the share of all images labeled correctly. The false positive rate is the share of human-made works wrongly called AI. The table reproduces the paper's results on unaltered images; the ensemble row combines three detectors.
| Judge | Accuracy | Human art wrongly flagged as AI |
|---|---|---|
| General online participants | 59.23 percent | 40.81 percent |
| Professional artists | 75.32 percent | 23.53 percent |
| Expert artists | 83.00 percent | 20.78 percent |
| Hive (detector) | 98.03 percent | 0.00 percent |
| Optic (detector) | 90.67 percent | 24.47 percent |
| Illuminarty (detector) | 72.65 percent | 67.40 percent |
| Ensemble of Hive, Optic, and Illuminarty | 98.75 percent | 0.48 percent |
Experienced viewers beat casual ones by a wide margin, so familiarity with AI output counts for a lot. Even so, the best human group still called about one human-made image in five an AI product, because ordinary human flaws can look like machine artifacts. Hive made no such error in this test.
What happens when an artist protects a style
Glaze is a free tool from the University of Chicago's SAND Lab that adds changes to an artwork that look minor to people but are meant to disrupt an AI model's attempt to copy the artist's style. The researchers tested how such filtered images affected detection. For Hive, the share of AI images correctly flagged dropped from 96.83 percent to 69.73 percent at medium Glaze strength and 67.56 percent at high strength. For Optic the drop was from 98.85 percent to 62.62 percent and 58.00 percent. Expert artists were steadier: they scored 83.44 percent on the filtered set, against 87.76 percent for Hive.
The combination worked best. Hive plus an expert reached 92.19 percent accuracy on unaltered images and 92.54 percent on filtered ones. The Glaze project page itself states the tool is not a permanent solution, and that methods used today could be overcome later.
Reading the results with care
Several limits apply. The test used a curated set of seven styles and five generators, so results may differ for newer models. The professional-artist group was recruited by survey, not randomly selected, and the expert group had only 13 people. Detector vendors update their products, so the 2024 numbers describe a past version. The authors Ben Zhao and Shawn Shan also lead the Glaze project, which is a conflict of interest the paper's readers should keep in view, even though the detector comparisons are reported in full tables.
The data supports a modest conclusion. One glance, one detector, or one expert does not settle whether a work is human-made or machine-made. When authenticity matters, as in a commission, a contest entry, or a sale, process files such as layered drafts or sketches add evidence that image analysis cannot supply. That advice is the editors' reading of the results, not a finding of the paper.
What has changed since the study
The Glaze project page, checked on October 27, 2026, names version 2.1 as the latest release, built to resist the noisy upscaler attack described in June 2024. It still says that Glaze is not a permanent solution against AI mimicry. In December 2025, three researchers posted a separate paper on arXiv (Martin, Chan, and Ma) reporting that Glaze and Nightshade add structured, content-linked changes that stay consistently detectable by analysis, and that applying both in sequence amplifies the detectable structure. That paper studies the protection filters, not the accuracy of people or AI-image detectors. No newer study with the same design (artists, online participants, and commercial detectors on the same image set) was found, so the 2024 percentages are the latest available figures. Detector vendors and image generators have changed since early 2024, and the numbers should not be assumed to hold for current versions.
Checklist for verifying a disputed image
- Ask for working files: layers, sketches, timestamps, and earlier versions.
- Compare against the artist's earlier portfolio for consistent style and skill.
- Run more than one detector, and treat any result as a signal rather than proof.
- Ask a person experienced with AI images, while allowing for false alarms.
- Check for embedded provenance records, and remember that their absence proves nothing.
FAQ
Can an AI detector prove an image was made by AI?
No. In the study the best detector was highly accurate on unaltered images, but accuracy dropped when the images were filtered or compressed in certain ways. A detector result is evidence, not proof.
Are artists better than ordinary people at spotting AI images?
Yes, on the study's evidence. Professional artists scored 75.32 percent and expert artists 83.00 percent, against 59.23 percent for general participants. Experts also made more false accusations against human work.
What does Glaze do?
Glaze is a free University of Chicago tool that alters an image so that AI models trained on it learn a distorted version of the artist's style. Its developers say it is not permanent protection.
Did the study cover current image generators?
It covered five generators available in early 2024. Newer systems were not tested, and as of October 2026 no follow-up with the same design was located, so the exact percentages should not be assumed to hold today.
Related reading
- How AI paintings are made
- Where AI and art meet
- Human creativity and AI side by side
- AI tools in the creative process
More in AI Basics.
Last verified: October 2026
Sources
- Ha A. Y. J., Passananti J., Bhaskar R., Shan S., Southen R., Zheng H., Zhao B. Y., Organic or Diffused (distinguishing human art from AI-generated images), Proceedings of ACM CCS 2024
- What is Glaze, SAND Lab, University of Chicago, checked October 2026
- Martin M. R., Chan G., Ma K.-L., Interpreting Structured Perturbations in Image Protection Methods for Diffusion Models, arXiv 2512.08329, December 2025