Three documented designs show how AI systems keep a short-term memory. A 2013 Atari-playing agent fed a neural network the last four screen frames, a 1997 recurrent design called long short-term memory carried a gated internal state across time steps, and the 2017 Transformer attends over the text it is given at once. In each case the memory covers a recent window of input, and nothing in that window becomes permanent knowledge on its own.
Key takeaways
- The 2013 deep Q-network paper applied its preprocessing to the last four frames of a game history and stacked them as the network's input.
- Long short-term memory (LSTM), published in Neural Computation in 1997, uses multiplicative gate units that open and close access to a stored signal, and learned to bridge time lags above 1,000 steps in the authors' tests.
- The Transformer architecture, introduced in 2017, relies on attention alone and dispenses with recurrence and convolutions.
- A system's short-term memory is the window it can use right now. Changing the model's weights takes a separate training step.
- The "limited memory" label is a loose teaching category, and the three designs below are better described by what they actually store.
Example one: four frames so a game agent can see motion
A single still image of a ball does not say which way the ball is moving. The deep Q-network paper by Volodymyr Mnih and colleagues, "Playing Atari with Deep Reinforcement Learning," posted on December 19, 2013, handled this by giving the network a short history. According to the paper, the preprocessing function was applied to the last four frames of a history, and the results were stacked to form the input to the Q-function. The authors tested the method on seven Atari 2600 games, where it outperformed all previous approaches on six and surpassed a human expert on three.
An illustrative walk-through, using made-up numbers, shows why stacking helps. If the ball sits at horizontal position 10 in the first of four frames and at 12, 14, and 16 in the next three, the stack as a whole reveals a speed of two units per frame to the right. Any single frame would show only a position. The agent remembers nothing between decisions beyond what the stack holds, so when a frame leaves the window, it is gone.
Example two: a gated cell that carries information forward
Recurrent networks read a sequence one step at a time and pass a state forward. Early versions struggled to learn from events many steps back. Sepp Hochreiter and Jurgen Schmidhuber's 1997 paper in Neural Computation, "Long short-term memory," describes the problem as insufficient, decaying error backflow and introduces a gradient-based method, LSTM, to address it. Units called constant error carousels hold a signal, and multiplicative gate units learn to open and close access to it. The abstract reports that LSTM could bridge time lags in excess of 1,000 discrete time steps in the authors' experiments, which used artificial data.
The memory here is a learned state that the network updates as it reads. It is short-term in the sense that it is rewritten during one pass through a sequence. What the network learned about how to use that state is stored in its weights, and those change only when it is trained again.
Example three: attention over the supplied text
The 2017 paper "Attention Is All You Need" proposed the Transformer, a network based solely on attention mechanisms, with recurrence and convolutions removed. It reported 28.4 BLEU on the WMT 2014 English-to-German translation task and 41.8 on English-to-French. A language model built on this idea works with the sequence of tokens it is handed. Text outside that sequence is unavailable to it unless a system supplies it again, which is why chat products have to decide what to include with each new request. The size of the usable sequence differs by model and is stated in each vendor's documentation.
Comparing the three designs
| Design | What counts as short-term memory | What is lost | Where lasting learning lives |
|---|---|---|---|
| Deep Q-network (2013) | The last four preprocessed frames | Anything older than four frames | Network weights, changed during training |
| LSTM (1997) | A gated internal state updated at each step | Information the gates close off or overwrite | Network weights, changed during training |
| Transformer (2017) | The token sequence the model attends over | Anything not included in the input | Network weights, changed during training |
Common mix-ups about "limited memory"
- A short-term window is not the same as the training data. Training shapes weights ahead of time, and the window is what the system uses at the moment of a decision.
- A larger window does not guarantee better use of what is in it, so a figure for capacity says little about reliability.
- Apps may add saved notes or profiles outside the model. That is a separate feature of the product and varies by vendor.
- The examples above come from research papers, and the commercial products that descend from them are built and changed in ways the papers do not describe.
FAQ
Is a four-frame history really "memory"?
It is a short-term record of recent inputs that the network uses when choosing an action. It does not persist after the frames are replaced, and the paper does not describe it as a store of experience.
Does LSTM remember for long periods?
The paper reports bridging time lags above 1,000 steps on artificial test tasks. That measures one design on synthetic data and does not set a limit for all uses.
Do chatbots built on Transformers learn from each conversation?
Within a conversation, earlier turns can be supplied as part of the input. The model's weights do not change from that alone. Whether a product saves anything between sessions depends on the product.
Why does the choice of window matter in practice?
A window that is too short leaves out information the task needs, such as the direction of motion in the Atari example. Choosing what to include is therefore a design decision and not a detail.
Related reading
- Limited memory AI and its constraints
- Machine learning with restricted data storage
- What artificial intelligence cannot do
- Artificial narrow intelligence, with examples
More in AI Basics.
Last verified: October 2026
Sources
- Mnih V et al., Playing Atari with Deep Reinforcement Learning, arXiv:1312.5602, NIPS Deep Learning Workshop, 2013
- Hochreiter S, Schmidhuber J, Long short-term memory, Neural Computation, volume 9, issue 8, 1997
- Vaswani A et al., Attention Is All You Need, arXiv:1706.03762, 2017