Most AI systems learn during a training phase and then stay fixed until developers retrain them, so "learning and adapting" in everyday use mostly means something narrower than it sounds. A language model can follow new instructions and examples placed in a prompt without any change to its internal weights, and the GPT-3 authors tested it "without any gradient updates or fine-tuning". Lasting change requires retraining, which brings its own risk: a network that learns a new task can lose skill at an old one.
Key takeaways
- Adapting inside a prompt, retraining on fresh data, and learning tasks in sequence are three different mechanisms with different limits.
- Catastrophic forgetting is the loss of earlier skills when a network is trained on new tasks, and a 2016 preprint tested one remedy on handwritten digits and Atari 2600 games.
- Concept drift is a change in the data over time that cannot be predicted in advance, and models that ignore it perform poorly.
- Google's machine learning course advises tracking how long it has been since a model was last retrained.
- Microsoft's Tay chatbot, built to improve through conversation, was taken offline after a coordinated attack within its first day.
Four things "learning" can mean
The question of whether AI truly learns has several honest answers because the word covers different processes. The table separates them by whether the model's weights, the numbers that encode what it has learned, change. The row on retraining reflects practice described in Google's course, and the others rest on the papers named in each row.
| Process | Weights change? | Who triggers it | Main limit |
|---|---|---|---|
| Initial training | Yes | Developers | Reflects only the data it was given |
| Prompting with examples (few-shot use) | No | The user, in the text | Lasts only as long as the prompt |
| Scheduled retraining on newer data | Yes | Developers | Quality can slip between updates |
| Learning new tasks one after another | Yes | Developers | Earlier skills may degrade |
Adapting inside a conversation
The 2020 paper on GPT-3, submitted to arXiv on May 28, 2020, reports that scaling up a language model improves its results on new tasks when it sees only a few examples. The tasks and demonstrations were specified purely as text, and the model was applied without gradient updates or fine-tuning. That is adaptation of a kind: the same fixed model behaves differently depending on what the prompt contains. It is not learning in the sense of permanent change, because once the prompt is gone the example is gone too, unless the product stores it elsewhere.
Why a network cannot simply keep learning
The obvious fix is to keep training a model on everything it meets. The trouble is that neural networks tend to lose performance on earlier tasks when they learn new ones in sequence. A team led by James Kirkpatrick addressed this in a paper first submitted on December 2, 2016, titled Overcoming catastrophic forgetting in neural networks. Their method slows learning on the weights that matter most for old tasks. They tested it on sequential classification of MNIST handwritten digits and on learning several Atari 2600 games one after another, and argued that the limitation can be overcome. The paper is a research result, and it does not mean that deployed products update themselves safely in this way.
When the world changes under a model
A model can be accurate on its test data and still decline in service, because the world moves. A 2020 review by Jie Lu, Joao Gama, and colleagues defines concept drift as shifts in the distribution of streaming data over time that cannot be predicted in advance. It breaks the response into detecting the drift, understanding it, and adapting to it, and warns that machine learning performs poorly under drift if the drift goes unaddressed.
Google's machine learning course, last updated October 16, 2025, gives practical advice. It says quality declines when serving data evolves, recommends tracking the time since a model was last retrained with an alert threshold, and describes serving a new version to a fraction of queries before shifting all traffic to it. In this view, adaptation is an engineering routine of monitoring and scheduled retraining rather than something the model does on its own.
A case of learning from users that went wrong
Tay was a chatbot Microsoft released for 18- to 24-year-olds in the United States. In a blog post dated March 25, 2016, Peter Lee wrote that the company expected Tay to improve through interaction, and that a coordinated attack exploited a vulnerability within the first 24 hours. Tay posted offensive content, and Microsoft took it offline, accepted responsibility for not anticipating the possibility, and said it would bring Tay back only when it could better anticipate malicious intent. The episode shows why many products separate what a model reads from what it permanently learns. Read the account in the Microsoft post.
Questions to ask when a product says it learns
- Do user inputs change the model's weights, or only the content of a stored profile or prompt?
- How often is the model retrained, and who decides when?
- Is there monitoring that would reveal a drop in quality as the data shifts?
- What filters stand between untrusted user input and any learning step?
- Can a bad update be rolled back?
FAQ
Does ChatGPT learn from a chat?
Within one conversation, the model uses the text it is given, which works like few-shot prompting. Whether a vendor later uses conversations for training is a policy matter that varies by product, and the privacy terms of the specific service settle it.
What is catastrophic forgetting?
It is the tendency of a neural network to lose performance on earlier tasks after learning new ones in sequence. The Kirkpatrick paper proposed slowing changes to the weights most important for old tasks.
Why do models get worse over time?
One cause is concept drift: the data the model meets in service differs from the data it was trained on. Monitoring and retraining are the standard responses.
Is few-shot prompting real learning?
It is adaptation without a weight change. The model's behavior shifts for the length of the prompt, but its stored parameters do not.
Related reading
- What the code behind an AI system does
- The math behind model training
- How far AI autonomy goes
- Implementing AI training step by step
More in AI Basics.
Last verified: October 2026
Sources
- Overcoming catastrophic forgetting in neural networks, Kirkpatrick et al., arXiv:1612.00796, 2016
- Language Models are Few-Shot Learners, Brown et al., arXiv:2005.14165, 2020
- Learning under Concept Drift: A Review, Lu et al., arXiv:2004.05785, 2020
- Production ML systems: Monitoring, Machine Learning Crash Course, Google for Developers, 2025
- Learning from Tay's introduction, Peter Lee, Microsoft, March 25, 2016