The history of modern AI can be read as five technical steps, each published in a dated paper: a mathematical neuron in 1943, a learning machine called the perceptron in 1958, back-propagation for multilayer networks in 1986, a deep network that won an image contest in 2012, and the Transformer architecture in 2017. Each step changed one ingredient (the model, the learning rule, the scale, or the design), and the order matters more than the dates. A closing section checks what the field looked like in 2026.
Key takeaways
- Warren McCulloch and Walter Pitts published a logical model of neurons in 1943, before the term "artificial intelligence" existed.
- Frank Rosenblatt's perceptron appeared in Psychological Review in November 1958, and back-propagation was published in Nature on October 9, 1986.
- In the 2012 ImageNet contest, a University of Toronto network reached a top-5 error of 15.3 percent against 26.2 percent for the runner-up.
- The Transformer paper was posted on arXiv on June 12, 2017, and reported 28.4 BLEU on English-to-German translation.
- Every date below is a publication or contest date, not the moment a technology became useful. The papers end in 2017; the section on 2026 covers what came after.
1943: neurons described as logic
McCulloch and Pitts, in the Bulletin of Mathematical Biophysics, argued that because nerve cells fire in an all-or-none way, neural events and the relations among them can be treated with propositional logic. They also showed that for suitable logical expressions a network of such units could be built to behave as the expression describes. The model existed only on paper and had no learning in it. Its lasting contribution was the idea that a brain-like network can be analyzed as a computing device, a starting point for later neural network research.
1958: a model of storage and organization
Rosenblatt's paper, published in Psychological Review in November 1958 under the title "The perceptron: a probabilistic model for information storage and organization in the brain," developed what its indexed record describes as a theory for a hypothetical nervous system named the perceptron, serving as a bridge between biophysics and psychology. The step beyond 1943 was that the perceptron was presented as a system whose organization depends on the information it receives, instead of a fixed logical circuit. The details of its training procedure are in the paper itself and are not summarized here.
1986: training networks with hidden layers
David Rumelhart, Geoffrey Hinton, and Ronald Williams described back-propagation in Nature on October 9, 1986. The procedure adjusts a network's weights repeatedly to reduce the difference between its actual output and the desired output, and the internal hidden units come to represent features of the task. According to the paper's abstract, the ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure. The 2012 network described below was likewise trained by stochastic gradient descent, which relies on this kind of error-driven weight adjustment.
2012: scale, data, and graphics processors
Alex Krizhevsky, Ilya Sutskever, and Hinton, all at the University of Toronto, trained a large convolutional neural network with 60 million parameters on two GTX 580 graphics processors; training took between five and six days. A variant entered in the ILSVRC-2012 image classification contest achieved a winning top-5 test error rate of 15.3 percent, against 26.2 percent for the second-best entry, as reported in their paper at the NIPS 2012 conference. The authors wrote that results could likely be improved by waiting for faster processors and larger datasets. What 2012 added, then, was mostly scale, not a new principle.
2017: attention replaces recurrence
Ashish Vaswani and seven co-authors posted "Attention Is All You Need" to arXiv on June 12, 2017. The paper proposed the Transformer, a network built on attention mechanisms alone. On the WMT 2014 English-to-German translation task it reached 28.4 BLEU, which the authors reported as over two BLEU above the best earlier results, including ensembles. On English-to-French it set a single-model record of 41.8 BLEU after 3.5 days of training on eight GPUs. The architecture is the basis of the large language models that became widely known after 2022, though the 2017 paper itself was about translation.
What each step changed
| Year | Paper | Ingredient that changed |
|---|---|---|
| 1943 | McCulloch and Pitts | A formal model of the neuron |
| 1958 | Rosenblatt | A probabilistic model of how a network stores and organizes information |
| 1986 | Rumelhart, Hinton, and Williams | Training of hidden layers |
| 2012 | Krizhevsky, Sutskever, and Hinton | Scale: large data, large network, GPUs |
| 2017 | Vaswani and colleagues | Architecture based on attention |
The table is a simplification. Many papers sit between these five, and other lines of work, such as the rule-based systems and search methods that dominated parts of the 1960s to the 1980s, are not included.
What changed after 2017: status in 2026
The five papers stop in 2017, so later events need their own sources. On October 8, 2024, the Royal Swedish Academy of Sciences gave the Nobel Prize in Physics to John Hopfield and Geoffrey Hinton, with a citation for foundational discoveries and inventions that enable machine learning with artificial neural networks. Hinton is the same Hinton who co-authored the 1986 and 2012 papers above.
Stanford's Institute for Human-Centered AI publishes the AI Index each year, and its 2026 report is the most recent edition as of October 2026. It states that organizational adoption of AI reached 88 percent and that scores on the SWE-bench Verified coding benchmark rose from 60 percent to near 100 percent in one year. These are the report's own headline claims, and benchmark scores in particular describe performance on a fixed test, not general ability. The Transformer of 2017 remains the architecture behind many language models, but this page does not rank newer designs or name a successor.
How to read a timeline like this one
- Check what a date marks: a paper, a product launch, or a demonstration.
- Look for what was new in the entry, since many milestones reuse older ideas at a larger scale.
- Note what the paper measured and on which data, because a contest result does not describe general ability.
- Treat the choice of milestones as an editor's decision. Another author could list different ones.
FAQ
When did AI begin?
There is no single start date. The 1943 model, the 1950s perceptron, and the field's naming in the mid-1950s are all used, depending on whether the question concerns ideas, machines, or the research community.
Was the 2012 result the invention of deep learning?
No. Multilayer networks and back-propagation were published in 1986. The 2012 result showed that, with enough data and graphics processors, such networks could win a major benchmark by a wide margin.
What is BLEU?
BLEU is a score that compares machine translation output with human reference translations. Higher is better, and scores from different tasks should not be compared directly.
Did the Transformer lead directly to chatbots?
It is the architecture underneath many current language models, but the 2017 paper dealt with translation. Later work in training, data, and product design was needed for chat systems.
Related reading
- Where AI technology began
- The roots of AI research
- AI's development through the years
- Other terms used for artificial intelligence
More in AI Basics.
Last verified: October 2026
Sources
- A logical calculus of the ideas immanent in nervous activity, McCulloch W S and Pitts W, Bulletin of Mathematical Biophysics 5, 1943
- The perceptron: a probabilistic model for information storage and organization in the brain, Rosenblatt F, Psychological Review 65(6), 1958
- Learning representations by back-propagating errors, Rumelhart D E, Hinton G E and Williams R J, Nature 323, 1986
- ImageNet Classification with Deep Convolutional Neural Networks, Krizhevsky, Sutskever, and Hinton, Advances in Neural Information Processing Systems 25, 2012
- Attention Is All You Need, Vaswani A et al., arXiv:1706.03762, June 2017
- The Nobel Prize in Physics 2024, press release, Royal Swedish Academy of Sciences, October 8, 2024
- The 2026 AI Index Report, Stanford Institute for Human-Centered AI, 2026