An artificial neural network is a function built from many small units, each of which multiplies its inputs by learned weights, adds a bias, and passes the total through a nonlinear activation. A single unit can only separate inputs with a straight line, so tasks such as XOR need at least one hidden layer. The numbers below can be checked by hand.
Key takeaways
- One artificial neuron computes a weighted sum plus a bias, then applies an activation function such as sigmoid or ReLU.
- A single neuron cannot compute XOR, because no straight line separates its two output classes; two hidden units fix this.
- Without a nonlinear activation, any number of stacked layers collapses into one linear map.
- A one-hidden-layer network can approximate any continuous function given enough units, but the theorem says nothing about how to find the weights.
- The brain comparison is loose: Stanford's CS231n notes call the neuron model very coarse.
What one artificial neuron does
Take two inputs, x1 = 2 and x2 = -1, with weights w1 = 0.5 and w2 = 0.8 and a bias of 0.1. The weighted sum is 0.5 times 2, plus 0.8 times -1, plus 0.1, which is 0.3. A sigmoid activation turns that into 1 / (1 + e^-0.3), about 0.574. That single number is the neuron's output. Training means adjusting the weights and bias so outputs match known answers more often.
The idea has old roots. McCulloch and Pitts described neurons as threshold logic units in 1943, and Rosenblatt published his perceptron model in Psychological Review in 1958. Modern units differ mainly in using smooth or piecewise-linear activations that suit gradient-based training.
Why one neuron is not enough: the XOR test
XOR outputs 1 when exactly one of two binary inputs is 1. A neuron with a step activation fires when w1 times x1 plus w2 times x2 plus b is above zero. Suppose it could compute XOR. Inputs (1,0) and (0,1) must fire, so w1 + b > 0 and w2 + b > 0. Input (0,0) must not fire, so b <= 0. Input (1,1) must not fire, so w1 + w2 + b <= 0. Adding the first two inequalities gives w1 + w2 + 2b > 0, so w1 + w2 + b > -b, which is at least zero. That contradicts the last condition. No weights exist.
Geometrically, the four input points cannot be split by one line. Adding a hidden layer changes the picture.
A two-layer solution you can verify
Use two hidden units with ReLU activation, which returns the larger of zero and its input. Let h1 = ReLU(x1 + x2) and h2 = ReLU(x1 + x2 - 1). Let the output be h1 - 2 times h2. The table shows every case. This is an illustrative hand-built example, not a trained network; training would find some other set of weights that does the same job.
| x1 | x2 | h1 | h2 | Output |
|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 |
| 1 | 0 | 1 | 0 | 1 |
| 0 | 1 | 1 | 0 | 1 |
| 1 | 1 | 2 | 1 | 0 |
This network has nine adjustable numbers: six weights (four into the hidden layer, two into the output) and three biases. Real networks scale the same pattern to millions or billions of such numbers.
Why the activation function matters
If every unit only computed a weighted sum, then a second layer would compute a weighted sum of weighted sums, which is still one weighted sum. Depth would add nothing. The Stanford CS231n course notes make this point and describe the nonlinearity as critical. They also list the common choices: sigmoid, which squashes values into the range 0 to 1 and can saturate; tanh, a scaled sigmoid centered on zero; and ReLU, the max(0, x) function, which the notes recommend as a default while warning that units can stop responding.
What the universal approximation result does and does not say
Hornik, Stinchcombe, and White showed in 1989 that networks with a single hidden layer and squashing activations can approximate a very wide class of functions, given enough hidden units. Read carefully, this is an existence statement. It does not say how many units are needed, how to find the weights from data, or whether the result will generalize to new examples. The CS231n notes call it a weak statement in practice. Networks succeed because the functions that matter tend to be smooth and because training methods can find usable weights, and the theorem alone explains neither.
The brain analogy and its limits
Neurons are often drawn as dendrites, a cell body, and an axon, and compared to the units above. The CS231n notes caution that the model is very coarse: real dendrites perform nonlinear computation, synapses involve more than a single number, and timing of spikes may matter. The comparison helps in explaining the vocabulary. It is a poor guide to predicting what a network can or cannot do.
FAQ
Is an artificial neural network the same as deep learning?
Deep learning refers to networks with many layers of learned representations, as described by LeCun, Bengio, and Hinton in Nature in 2015. The XOR network above is a neural network but is too shallow to be called deep.
Does the network learn the weights by itself?
Weights start with arbitrary values and are adjusted by an optimization procedure that reduces the error on training examples. The designer still chooses the architecture, the data, the error measure, and the stopping point.
Why can a network with enough units still fail?
The approximation theorem concerns what is representable, not what is learnable from limited data. A network can memorize training examples and perform poorly on new ones, so performance must be checked on held-out data.
Do neural networks work like the human brain?
Only loosely. The units borrow terminology from neuroscience, but the mathematics is a stack of weighted sums and simple nonlinear functions, and the training method has no known counterpart in the brain.
Related reading
- Applications and functions of neural networks
- How vectors are used in AI
- Machine learning methods beyond neural networks
- Framing a problem for an AI system
More in AI Basics.
Last verified: October 2026
Sources
- Neural Networks Part 1: Setting up the Architecture, CS231n course notes, Stanford University
- Multilayer feedforward networks are universal approximators, Hornik K, Stinchcombe M, White H, Neural Networks 2(5), 1989
- The perceptron: a probabilistic model for information storage and organization in the brain, Rosenblatt F, Psychological Review 65(6), 1958
- A logical calculus of the ideas immanent in nervous activity, McCulloch WS, Pitts W, Bulletin of Mathematical Biophysics 5, 1943
- Deep learning, LeCun Y, Bengio Y, Hinton G, Nature 521, 2015