Artificial neural networks for image recognition borrow their layered design from studies of the brain, yet they can reach the same labels by a different route. In a 2019 test that pitted conflicting texture and shape cues against each other, human observers chose the shape category for 95.9 percent of their correct answers, while a standard ResNet-50 network chose shape for only 22.1 percent. The difference shows that matching a human's answers on ordinary photographs does not mean a model uses the same information.
Key takeaways
- Hubel and Wiesel's 1959 recordings from single neurons in the cat's visual cortex were echoed in Kunihiko Fukushima's 1980 neocognitron, which he described as resembling their hierarchy model.
- In a cue-conflict test, four ImageNet-trained networks mostly answered by texture, and humans mostly answered by shape.
- Training the same ResNet-50 on stylized images raised its shape responses from 22 percent to 81 percent.
- A 2021 follow-up found that the largest models matched humans better on distorted images but still made different mistakes.
- The comparison covers vision only and says nothing about memory, language, or reasoning.
The design came from the visual cortex
David Hubel and Torsten Wiesel published "Receptive fields of single neurones in the cat's striate cortex" in The Journal of Physiology in October 1959. They shared the 1981 Nobel Prize in Physiology or Medicine for discoveries about information processing in the visual system. The Nobel Assembly's announcement of October 9, 1981 summarized the finding this way: the signal from the eyes is analyzed in steps, with cells responding to contrasts, linear patterns, and the movement of an image, and with cells arranged in columns.
That staged picture shaped early network designs. Kunihiko Fukushima's 1980 paper in Biological Cybernetics described the neocognitron, a network with alternating layers of S-cells and C-cells, resembling simple and complex cells. He wrote that, after self-organization, the network had a structure similar to the hierarchy model of Hubel and Wiesel, and that its output was not affected by the position of a pattern. The convolutional networks tested in 2019 are also stacks of layers, in which early layers respond to small local patterns and later layers combine them.
Where the similarity stops: a cue-conflict test
Robert Geirhos and colleagues asked what such a network actually uses to recognize an object. Their paper, accepted at ICLR 2019, ran nine experiments with 97 human observers across 48,560 trials. In the key experiment, each picture combined the shape of one category with the texture of another, such as a cat outline filled with elephant skin. Both people and networks had to name one category, and the answer reveals which cue they relied on.
| Observer | Chose shape | Chose texture |
|---|---|---|
| Human observers | 95.9 percent | 4.1 percent |
| AlexNet | 42.9 percent | 57.1 percent |
| GoogLeNet | 31.2 percent | 68.8 percent |
| ResNet-50 | 22.1 percent | 77.9 percent |
| VGG-16 | 17.2 percent | 82.8 percent |
The percentages are shares of correct-category decisions in that experiment. The human texture figure in the table is the remainder of the 95.9 percent shape share. The networks were also far less accurate than people on silhouettes and edge drawings, where texture is absent.
Changing the training data changes the bias
The authors suspected the training task itself. ImageNet can be solved fairly well from small local patches, so a network has little reason to learn global shape. To test the idea, they trained a ResNet-50 on Stylized-ImageNet, a version of the dataset in which the original textures were replaced with the style of randomly chosen paintings. On this harder dataset the network reached 79.0 percent top-5 accuracy, compared with 92.9 percent for the same architecture on regular ImageNet. Its shape responses in the cue-conflict test rose from 22 percent to 81 percent, close to human behavior in many categories. The paper also reports greater robustness to several image distortions.
The result points to the data and the objective rather than the architecture alone. The same layered design can produce a texture reader or a shape reader depending on what it was trained to do.
A later check on whether the gap closed
In a NeurIPS 2021 paper, Geirhos and coauthors collected 85,120 trials from 90 human participants on 17 out-of-distribution datasets and evaluated models trained in different ways, with training sets from one million to one billion images. They report that the gap in robustness to distorted images is closing, and that the best models exceed human performance on most of these datasets in a quick, feedforward setting. A gap remains at the level of individual images: humans and models make different errors, while most models agree with each other. Consistency with humans often improved as the training set grew by one to three orders of magnitude.
Together the two papers suggest a narrow conclusion. A model can match a human on accuracy and still fail in different places, which matters when a system is used where mistakes carry costs.
Reading claims about brain-like AI
- Does the claim name a specific task, or does it compare "intelligence" in general?
- Is the comparison to human accuracy only, or also to human error patterns?
- Which model generation and training set were tested, and in what year?
- How many human participants were measured?
- Does the study test unusual inputs, or only photographs similar to the training data?
FAQ
Is a convolutional neural network a model of the brain?
It borrows a layered idea from studies of the visual cortex, but it is an engineering system. The cue-conflict results show that it can use different cues from people for the same task.
Why do networks rely on texture?
The authors argue that the standard ImageNet task can be solved using local texture evidence, so a network has little pressure to learn overall shape. When texture was made uninformative, shape responses rose.
Do newer models still show the same bias?
The 2019 test used networks from that period. The 2021 study found the robustness gap with humans closing for the best models, but it also found different error patterns, so the 2019 numbers should not be applied to current systems without a new test.
Does this show that AI cannot match the brain?
No. It shows a measured difference in how one kind of model recognizes images. It does not address other abilities, and results can change with new training methods.
Related reading
- Natural and machine intelligence compared
- Whether AI could surpass human intelligence
- Reviewing AI that integrates human reasoning
- The debate over replacing human intelligence
More in AI Basics.
Last verified: October 2026
Sources
- Receptive fields of single neurones in the cat's striate cortex, Hubel DH, Wiesel TN, The Journal of Physiology 148(3), October 1959
- The Nobel Prize in Physiology or Medicine 1981, press release, Nobel Assembly at Karolinska Institutet, October 9, 1981
- Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Fukushima K, Biological Cybernetics 36, 1980
- ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness, Geirhos R et al., ICLR 2019 (arXiv version 3, 2022)
- Partial success in closing the gap between human and machine vision, Geirhos R et al., NeurIPS 2021