Computer vision is the branch of artificial intelligence that lets software interpret images and video, most often by training neural networks on large sets of labeled examples. It is used in medical screening, crop disease diagnosis, wildlife monitoring, and tools for blind and low-vision users, and it works reliably only when new images resemble the data the model was trained on.
Key takeaways
- A digital image is a grid of numbers, and a vision model learns which patterns of numbers match labels such as "tumor," "diseased leaf," or "zebra."
- In a Swedish trial of 106,000 women, AI-supported mammography screening produced 12% fewer interval cancers and cut radiologists' reading workload by 44%.
- A neural network trained on 3.2 million camera-trap images identified animals with over 93.8% accuracy, saving an estimated 8.4 years of manual labeling.
- Accuracy can fall sharply for groups or settings missing from the training data, so local testing matters.
How a computer reads an image
A photograph is stored as a grid of pixel values. A vision model does not see objects; it learns statistical links between patterns in those values and human-provided labels. Early layers of a convolutional neural network respond to simple features such as edges, and deeper layers combine them into shapes and parts. The 2012 AlexNet paper, trained on the ImageNet dataset, used five convolutional layers and about 60 million parameters to sort images into 1,000 categories, and reported a top-5 error of 18.9% on the 2010 test data. The annual ImageNet challenge, the benchmark behind that result, has run since 2010.
From convolutional networks to transformers
Convolutional networks dominated until 2020, when Dosovitskiy and colleagues showed that a transformer, the architecture behind most language models, can classify images when it processes them as sequences of small patches. Their Vision Transformer achieved strong results against leading convolutional networks after pre-training on large datasets, while needing substantially fewer computing resources to train. Current systems often combine vision with language, so one model can answer questions about a photo. Whichever architecture is used, the basic dependence on labeled or paired training data remains.
Medical imaging: screening support and regulation
Radiology is where vision AI is most visible in health care. The MASAI trial at Lund University, published in The Lancet in 2026, randomized 106,000 women in Sweden aged 40 to 74 to AI-supported or standard screening. AI support was linked to 12% fewer interval cancers (82 cases against 93) and a 44% lower screen-reading workload. An earlier milestone came in April 2018, when the US Food and Drug Administration authorized IDx-DR, a system that screens retinal images for diabetic retinopathy without a clinician reading them.
Regulators treat these tools as medical devices. Health Canada's pre-market guidance for machine learning-enabled medical devices asks manufacturers to show that training data are adequately representative of the Canadian population and to report performance for relevant subgroups. This article is general information, not medical advice.
Crop disease diagnosis on a phone
PlantVillage Nuru is a smartphone app that uses a deep learning model to spot cassava mosaic disease, cassava brown streak disease, and green mite damage. It works without an internet connection. A 2020 study in Frontiers in Plant Science (Mrisho and colleagues) trained it on 2,756 annotated leaf images from coastal Tanzania. Nuru identified symptoms with 65% accuracy, against 40% to 58% for extension agents and 18% to 31% for farmers, and reached 74% to 88% when six leaves per plant were assessed. These results cover one crop group and one region.
Wildlife monitoring with camera traps
Camera traps produce millions of photographs, most of them empty. Norouzzadeh and colleagues (PNAS, 2018) trained deep networks on the 3.2-million-image Snapshot Serengeti dataset covering 48 species. The system identified animals with over 93.8% accuracy. When it labeled only images it was confident about, it automated 99.3% of the data at 96.6% accuracy, matching crowdsourced volunteers, and the authors estimate this saves more than 8.4 years of labeling at 40 hours a week, or over 17,000 hours.
Accessibility: describing the visual world
Seeing AI is a free app, built with and for blind and low-vision users, that uses a phone camera and AI to narrate the world: it reads text, describes photos, and identifies products. Tools of this kind combine image recognition with speech output, so the quality of the description depends on image quality and on how well the scene matches what the model has seen. Because descriptions can contain errors, safety-related uses such as reading medication labels call for a second check, for example from a pharmacist.
Documented examples compared
| Application | Reported result | Stated limit |
|---|---|---|
| Breast screening (MASAI trial, Sweden) | 12% fewer interval cancers; 44% less reading workload | One country, one screening program |
| Cassava disease (Nuru, Tanzania) | 65% accuracy per image; 74% to 88% with six leaves | One crop, one region, trained on 2,756 images |
| Camera traps (Snapshot Serengeti) | Over 93.8% accuracy on 48 species | One habitat; new sites may differ |
| Image description (Seeing AI) | Reads text, describes photos, identifies products; free | No published accuracy figure cited; descriptions may be wrong |
Where computer vision fails
Failures usually trace back to the training data. A 2018 audit of three commercial systems that guess a person's gender from a face photo (Gender Shades) found the worst error rate, 34.7%, for darker-skinned women and the best, 0.8% or less, for lighter-skinned men. Real deployments also drift: the WILDS collection of ten datasets, built from cases such as images taken at different hospitals or by different camera traps, shows that standard training leaves models noticeably less accurate on unfamiliar data. A model built for one camera, lighting condition, population, or location should therefore be re-tested before it is used in another.
FAQ
How does computer vision differ from image processing?
Image processing changes an image, for example by sharpening or resizing it. Computer vision extracts meaning from it, such as which objects are present or whether a scan shows an abnormality.
Does vision AI replace doctors or ecologists?
The cited examples position the software as support. In the MASAI trial, radiologists remained part of the screening process, and in the Serengeti study, experts and volunteers provided the labels the system learned from.
Why does a model work in one place and fail in another?
The model learns patterns specific to its training images. A change in camera, lighting, patient population, or environment moves the inputs away from those patterns, which is the distribution shift measured in the WILDS benchmark.
Is facial recognition the same as computer vision?
Facial recognition is one application of computer vision. The same techniques also power crop diagnosis, medical imaging, and wildlife surveys.
Related reading
- AI applications in agriculture
- How deep learning works
- Recent advances in AI
- What artificial intelligence is
More in AI Basics.
Last verified: October 2026
Sources
- Krizhevsky, Sutskever, Hinton, ImageNet Classification with Deep Convolutional Neural Networks, NeurIPS, 2012
- Russakovsky et al., ImageNet Large Scale Visual Recognition Challenge, arXiv:1409.0575, 2014
- Dosovitskiy et al., An Image is Worth 16x16 Words, arXiv:2010.11929, 2020
- Lund University, MASAI trial results, The Lancet, 2026
- US Food and Drug Administration, FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems, 2018
- Health Canada, Pre-market guidance for machine learning-enabled medical devices, 2026
- Mrisho et al., Accuracy of a Smartphone-Based Object Detection Model, PlantVillage Nuru, Frontiers in Plant Science, 2020
- Norouzzadeh et al., Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning, PNAS, 2018
- Microsoft, Seeing AI app description, 2026
- Buolamwini and Gebru, Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification, 2018
- Koh and colleagues, benchmark of real-world distribution shifts (WILDS), arXiv:2012.07421, 2020