GPT-3 is a neural network with 175 billion parameters, described in a paper by OpenAI researchers dated May 28, 2020. It generates text by predicting the next token, and instructions or examples typed into the prompt steer it without retraining for each task. OpenAI opened a private-beta API for it on June 11, 2020, and Microsoft announced an exclusive license on September 22, 2020. This page keeps the 2020 record and adds what has happened to the model, the API, and the license by October 2026.
Key takeaways
- The paper "Language Models are Few-Shot Learners" reports eight model sizes, from 125 million to 175 billion parameters, each trained on 300 billion tokens.
- Filtered web text made up 60 percent of the training mix, and the authors flag benchmark contamination from web data as a methodological concern.
- Readers identified GPT-3 news articles of about 500 words with 52 percent accuracy, barely above the 50 percent chance level.
- The original GPT-3 models in OpenAI's API were retired on January 4, 2024. Their two replacements are scheduled to follow on September 28, 2026.
- Microsoft's license to OpenAI models and products now runs through 2032 and, since April 27, 2026, is non-exclusive.
What the 2020 paper built
The GPT-3 paper on arXiv lists 31 authors and describes an autoregressive model: it reads a sequence of tokens and predicts the next one. According to the abstract, the model is roughly ten times larger than any earlier non-sparse language model. The central claim concerns method. When tasks are specified only through text in the prompt, with no gradient updates and no fine-tuning, performance improved as the model grew.
The authors test three settings. Zero-shot gives the model a task description alone, one-shot adds a single worked example, and few-shot adds as many examples as fit in the context window of 2,048 tokens. All eight models share that window. The largest has 96 layers and the smallest has 12.
Training data and compute, in numbers
The data table shows that the model did not read its sources equally. Sets judged higher in quality were sampled more often than their size alone would suggest, which the authors describe as accepting a little overfitting in return for quality. Common Crawl, a public archive of web pages, was filtered from 45 terabytes of compressed text to 570 gigabytes, drawn from monthly snapshots between 2016 and 2019.
| Dataset | Tokens in dataset | Share of training mix |
|---|---|---|
| Common Crawl (filtered) | 410 billion | 60 percent |
| WebText2 | 19 billion | 22 percent |
| Books1 | 12 billion | 8 percent |
| Books2 | 55 billion | 8 percent |
| an online encyclopedia | 3 billion | 3 percent |
Every model saw 300 billion tokens in total, so Encyclopedia text was repeated up to 3.4 times while Common Crawl was seen less than once. The paper warns that web-scale pretraining can let benchmark test sets leak into training data. The authors searched for such overlaps and tried to remove them. A bug in the filtering let some through, and retraining was too costly, so the paper measures the effect of the remaining overlap in a separate analysis.
How it reached developers in 2020
OpenAI announced its API on June 11, 2020. News coverage reported that access went through a private beta with a request form. OpenAI's stated reason for not publishing the models openly was that an API lets the company widen access gradually and respond if harmful uses appear. A dozen or so partner companies were involved, and none were named.
On September 22, 2020, Microsoft chief technology officer Kevin Scott wrote that Microsoft had licensed GPT-3 exclusively for use in its own products and services. OpenAI would keep offering GPT-3 and other models through its own API, so the license covered the underlying model and not access through that interface.
Limits the authors documented
The limitations section is candid. Generated passages sometimes repeat themselves, lose coherence over long stretches, or contradict earlier sentences. The authors point to trouble with common-sense physics, such as whether cheese left in a refrigerator will melt. On comparison tasks, such as judging whether a word carries the same meaning in two sentences, few-shot results were little better than chance.
The bias analysis used prompts like "The detective was a" and counted which gender words the model made likely. Of 388 occupations tested, 83 percent were more likely to be followed by a male identifier. The authors say models trained on internet text carry internet-scale biases and call their own analysis preliminary.
The human-detection experiment points the other way. For articles of about 500 words, participants separated GPT-3 output from real news at 52 percent accuracy, against 88 percent for a deliberately weak control model. The result reflects one set of news-style prompts and about 80 US-based participants per experiment, so it does not extend to every kind of text.
What changed since 2020
An OpenAI paper submitted on March 4, 2022, on training with human feedback to follow instructions, tested a different approach. Human evaluators preferred outputs from InstructGPT, a model fine-tuned on human demonstrations and rankings, to those of the 175 billion parameter GPT-3, even when the instruction-tuned model had only 1.3 billion parameters. The authors also report gains in truthfulness and fewer toxic outputs, and note that the models still make simple mistakes.
The product side moved faster than the research record. The table collects the dated changes that could be checked against primary posts as of October 27, 2026.
| Date | Change | Source type |
|---|---|---|
| January 4, 2024 | The API models ada, babbage, curie, and davinci shut down; babbage-002 and davinci-002 were named as replacements | OpenAI deprecations page |
| August 7, 2025 | OpenAI announced GPT-5 for ChatGPT users and developers | OpenAI announcement |
| October 28, 2025 | Microsoft and OpenAI announced that Microsoft's IP rights for models and products run through 2032 | Microsoft blog |
| April 27, 2026 | Microsoft's license to OpenAI IP for models and products became non-exclusive; revenue share payments from OpenAI to Microsoft continue through 2030 | Microsoft blog |
| September 28, 2026 | Scheduled shutdown of babbage-002 and davinci-002; fine-tuned versions are listed for October 23, 2026 | OpenAI deprecations page |
In practice, the models described in the 2020 paper can no longer be called through OpenAI's API. The April 2026 Microsoft post speaks of OpenAI IP for models and products in general terms, not of GPT-3. Anyone relying on a specific legacy model name should read OpenAI's current deprecations page, because the dates above are scheduled changes that OpenAI can amend.
FAQ
Is GPT-3 the same thing as ChatGPT?
No. GPT-3 is the 2020 model family. ChatGPT, released in November 2022, is a chat product built on later models fine-tuned with human feedback.
Can GPT-3 still be used through the API?
The original GPT-3 models in the API (ada, babbage, curie, davinci) were retired on January 4, 2024. Their replacements, babbage-002 and davinci-002, were scheduled for shutdown on September 28, 2026. The 2020 paper itself remains available on arXiv.
What does 175 billion parameters mean?
A parameter is a learned numerical weight inside the network. The count measures size, not accuracy or knowledge, and the paper shows that more size helps on many tasks but not all.
What is few-shot learning in this context?
It means placing a handful of worked examples inside the prompt. The model's weights do not change. The authors say it is unclear whether this teaches a new task or only helps the model recognize one it already learned in training.
Could people tell GPT-3 text from human text?
In the paper's news-article tests, accuracy for the largest model was close to a coin flip. The authors add that factual errors, repetition, and odd phrasing can still give generated text away.
Related reading
- Google's LaMDA language model
- A later OpenAI release, GPT-4.5
- Natural language generation in chat bots
- Alternatives to OpenAI for language processing
More in AI Tools and Industry.
Last verified: October 2026
Sources
- Brown and 30 co-authors, the GPT-3 paper (arXiv:2005.14165), OpenAI researchers, 2020
- Ouyang and co-authors, the InstructGPT paper (arXiv:2203.02155), OpenAI researchers, 2022
- Coldewey, OpenAI makes an all-purpose API for its text-based AI capabilities, June 11, 2020
- Scott, Microsoft teams up with OpenAI to exclusively license GPT-3 language model, Microsoft Official Blog, September 22, 2020
- Deprecations, OpenAI developer documentation, accessed October 2026
- Introducing GPT-5, OpenAI, August 7, 2025
- The next chapter of the Microsoft-OpenAI partnership, Microsoft Official Blog, October 28, 2025
- The next phase of the Microsoft-OpenAI partnership, Microsoft Official Blog, April 27, 2026