Deep learning is machine learning that uses neural networks with many processing layers, each learning its own representation of the data, with the internal parameters adjusted by an algorithm called backpropagation. A 2012 network co-authored by Geoffrey Hinton, with 60 million parameters, reached a top-five error rate of 18.9 percent on the ImageNet 2010 test data, considerably better than earlier results. Canada has a direct link to the field: Hinton was a professor at the University of Toronto when he shared the 2024 Nobel Prize in Physics, and Yoshua Bengio, his co-author on a landmark review, was at the Université de Montréal.
Key takeaways
- "Deep" refers to the number of layers: each layer builds on the one before it, moving from simple to more abstract features.
- Training means repeatedly nudging parameters to reduce error on examples, and the 2012 ImageNet network had 60 million of them.
- The Nobel Prize in Physics 2024, announced October 8, 2024, went to John Hopfield and Geoffrey Hinton for foundational work on artificial neural networks.
- Deep networks can be fooled by small, deliberately chosen changes to an input, so high test accuracy is not the same as reliability.
- A deep network learns from data, so its quality depends on the examples it was trained on.
What "deep" means
A neural network is a set of simple units that take numbers in, multiply them by adjustable weights, and pass a result on. In a deep network these units are stacked in layers. The 2015 review in Nature by Yann LeCun, Yoshua Bengio, and Geoffrey Hinton defines deep learning as computational models composed of multiple processing layers that learn representations of data with multiple levels of abstraction. The first layer might respond to tiny patterns in an image, and a later layer combines earlier outputs into something more complex. The designer does not hand-write these features; the training process finds them.
The 2012 image classifier is a concrete case. Its authors, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, describe five convolutional layers, some followed by max-pooling layers, then two fully connected layers and a final 1000-way output, with 500,000 neurons and 60 million parameters overall. It was trained on 1.3 million high-resolution images from the ImageNet set, sorted into 1000 classes.
How a network learns, in one small calculation
Training is repeated error correction. The example below is deliberately tiny and is an illustrative example, not taken from any paper: one input, one weight, no hidden layers.
| Step | What happens | Numbers |
|---|---|---|
| 1. Predict | Multiply input x by weight w | x = 2, w = 0.5, prediction = 1.0 |
| 2. Measure error | Compare with the target and square the gap | Target = 3, error = (1.0 - 3)^2 = 4 |
| 3. Find the direction | Compute how the error changes as w changes | 2 x (1.0 - 3) x 2 = -8 |
| 4. Update | Move w against the slope, scaled by a learning rate of 0.05 | w = 0.5 - 0.05 x (-8) = 0.9 |
| 5. Repeat | Predict again with the new weight | Prediction = 1.8, error = 1.44 |
The error fell from 4 to 1.44 in one step. A real network repeats this over millions of examples and adjusts millions of weights at once. Backpropagation is the algorithm the Nature review names for indicating how the machine should change the internal parameters in each layer.
Why the 2012 result mattered
The authors of the 2012 paper report top-one and top-five error rates of 39.7 percent and 18.9 percent on the ImageNet 2010 test data. Top-five means the correct label appears among the model's five best guesses. They credit a very efficient GPU implementation for making training fast enough, and a new regularization method for reducing overfitting. Three ingredients came together: a large labeled dataset, graphics processors suited to the arithmetic, and a network design able to use both.
By 2015 the Nature review could say that deep convolutional networks had brought breakthroughs in processing images, video, speech, and audio, while recurrent networks had done the same for sequential data such as text and speech.
The Canadian thread
The Royal Swedish Academy of Sciences awarded the Nobel Prize in Physics 2024 to John Hopfield of Princeton University and Geoffrey Hinton of the University of Toronto "for foundational discoveries and inventions that enable machine learning with artificial neural networks." The citation concerns the Hopfield network and Hinton's Boltzmann machine, early learning networks built with tools from physics. It does not describe the 2012 image classifier. In the 2015 Nature review, Bengio's affiliation is the Department of Computer Science and Operations Research at the Université de Montréal.
Where deep learning breaks
- Adversarial inputs: researchers reported in 2013 that an imperceptible perturbation, found by maximizing the network's prediction error, can make a network misclassify an image, and that the same perturbation can fool a different network trained on a different subset of the data.
- Opacity: the same paper notes that such networks learn solutions that are hard to interpret.
- Confident errors: a 2014 analysis found that several models misclassify adversarial examples with high confidence.
- Data dependence: the 2012 network was trained on 1.3 million labeled images, so its quality is tied to its training data.
Common misconceptions
- Deep learning is not the same as artificial intelligence: it is one family of methods inside machine learning.
- A network is not a copy of the brain. The Nobel announcement says the technology was originally inspired by the structure of the brain.
- More layers do not guarantee better results; data and training method matter as much.
- A network that scores well on a benchmark has not been shown to be safe in a new setting.
FAQ
What is the difference between machine learning and deep learning?
Machine learning covers any method that fits a model to data. Deep learning is the subset that uses networks with many layers and learns its own features instead of relying on hand-built ones.
Why is it called deep?
The name refers to the stack of processing layers. Each layer transforms the output of the one before, so the network represents the data at several levels of abstraction.
Did Geoffrey Hinton win the Nobel Prize for deep learning?
The 2024 Nobel Prize in Physics recognized foundational discoveries and inventions that enable machine learning with artificial neural networks, shared with John Hopfield. The Academy's text names the Boltzmann machine as Hinton's contribution.
Can a deep network explain its decisions?
Not by default. Researchers have described the solutions such networks learn as hard to interpret.
Related reading
- Computer vision in practice
- Artificial neural networks and their uses
- AI and deep learning compared
- What artificial intelligence is
More in AI Basics.
Last verified: October 2026
Sources
- LeCun Y, Bengio Y, Hinton G, Deep learning, Nature 521:436-444, May 27, 2015
- Krizhevsky A, Sutskever Ilya, Hinton GE, ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems 25, 2012
- Press release: The Nobel Prize in Physics 2024, Royal Swedish Academy of Sciences, October 8, 2024
- Szegedy C et al., Intriguing properties of neural networks, arXiv:1312.6199, 2013
- Goodfellow IJ, Shlens J, Szegedy C, Explaining and Harnessing Adversarial Examples, arXiv:1412.6572, 2014