LaMDA, short for Language Model for Dialogue Applications, is a family of language models that Google built for open-ended conversation. Google announced it on May 18, 2021, and a research paper submitted on January 20, 2022 described versions of up to 137 billion parameters, pre-trained on 1.56 trillion words. A lightweight version powered the first release of Google's Bard chatbot in February 2023, and Google later moved on to other models.
Key takeaways
- LaMDA was trained on dialogue as well as general web text, and its paper reports 92.3 percent sensibleness for the largest fine-tuned model.
- Grounding in outside sources was weaker: the paper reports 73.2 percent groundedness and 65 percent citation accuracy.
- Half of the pre-training data came from public forum dialogue, and more than 90 percent was English.
- Google said in June 2022 that a review found the evidence did not support a former engineer's claim that LaMDA was sentient.
- Bard started on LaMDA in February 2023, and the product was renamed Gemini on February 8, 2024.
What Google said in 2021
The May 18, 2021 announcement called LaMDA a research breakthrough in conversation technology. It was built on the Transformer neural network design that Google Research had released as open source in 2017, but, unlike most language models, it was trained on dialogue. Google said that training helped it follow open-ended conversations that drift from topic to topic. The company named two quality measures, sensibleness (does a reply make sense in context) and specificity (does it relate clearly to what was said), and said it was also exploring "interestingness" and working on factual accuracy. It added that reducing risks such as unfair bias, hateful speech, and misleading information was a top priority and that development had been reviewed against its AI Principles.
What the research paper reports
The paper, "LaMDA: Language Models for Dialog Applications" (arXiv:2201.08239), lists 60 authors, led by Romal Thoppilan. It describes models from two billion to 137 billion parameters, a decoder-only Transformer, and pre-training on a dataset Google called Infiniset. The figures below come from the paper's text.
| Item | Reported value |
|---|---|
| Largest model | 137 billion non-embedding parameters |
| Pre-training data | 2.97 billion documents, 1.12 billion dialogs, 13.39 billion utterances, 1.56 trillion words |
| Data mix | 50 percent public forum dialogue, 12.5 percent C4, 12.5 percent code-related pages, 12.5 percent English-language encyclopedia text, 6.25 percent other English web text, 6.25 percent non-English web text |
| Training compute | 1,024 TPU v3 chips for about 57.7 days |
| Sensibleness after fine-tuning | 92.3 percent |
| Groundedness and citation accuracy | 73.2 percent and 65 percent |
The authors found that making a model larger improved quality, but improved safety and factual grounding much less. Fine-tuning on crowdworker-annotated data and letting the model consult outside tools, such as a retrieval system, a translator, and a calculator, helped more. Even so, they reported that the models were still far from crowdworker performance on safety and groundedness.
Limits the authors stated
The paper is unusually direct about what remained unsolved. Grounding does not guarantee accuracy, and progress was limited to simple factual questions. The model could promise future answers, end conversations early, or invent details about the user. Safety filtering did not guarantee reliability, and representational harms could persist even when single replies passed checks. The safety objectives reflect a U.S. social context and cannot be universal, and crowdworkers were overrepresented in the 25 to 34 age group. The authors also flagged risks of impersonation and of people mistaking the system for a human.
The sentience claim of 2022
In June 2022, a Google engineer named Blake Lemoine, who had worked with LaMDA since 2021, said the system was sentient and shared transcripts of his conversations with it. Google put him on paid leave on June 13, 2022 for breaching its confidentiality policy. Spokesperson Brian Gabriel said a team of ethicists and technologists reviewed the concerns and found the evidence did not support them, and that today's conversational models are not sentient. Google dismissed Lemoine on July 22, 2022, according to reporting that day, citing violations of employment and data security policies. The episode is mostly a lesson in how fluent text can invite belief, a point the LaMDA paper itself raised when it warned about people mistaking such systems for humans.
From research model to product and successors
| Date | Event |
|---|---|
| August 25, 2022 | AI Test Kitchen app announced, using the latest LaMDA; access for small groups in the U.S., Android first |
| February 6, 2023 | Bard announced as an experimental service on a lightweight version of LaMDA, open first to trusted testers |
| May 10, 2023 | PaLM 2 introduced; Google said its multilingual abilities let Bard expand to new languages |
| February 8, 2024 | Bard renamed Gemini, with Gemini Pro 1.0 behind the standard experience |
Google's later announcements name PaLM 2 and Gemini, not LaMDA, and no retirement notice for LaMDA was found.
FAQ
What does LaMDA stand for?
Language Model for Dialogue Applications. The 2022 paper title uses "Dialog" in its spelling, while Google's 2021 announcement uses "Dialogue."
Was LaMDA the same thing as Bard?
No. LaMDA was the language model, and Bard was the chatbot service. Bard launched on a lightweight version of LaMDA, and Google later used other models behind it.
Was LaMDA sentient?
Google's internal review concluded that the evidence did not support that claim, and the company said conversational models of this kind are not sentient. The claim came from one engineer, and the paper itself makes no such statement.
Can the public download LaMDA?
No release of its weights was found in the sources reviewed. Access came through Google's own demos, such as AI Test Kitchen, and then through Bard.
Related reading
- Google's Gemini announcement
- OpenAI's GPT-3 language model
- DeepMind's AlphaGo
- Cohere and enterprise language models
More in AI Tools and Industry.
Last verified: October 2026
Sources
- Announcement of LaMDA conversation technology, Google blog, May 18, 2021
- Thoppilan R et al., LaMDA: Language Models for Dialog Applications, arXiv:2201.08239, January 2022
- Join us in the AI Test Kitchen, Google, August 25, 2022
- Google AI updates: Bard and new AI features in Search, Google, February 6, 2023
- Introducing PaLM 2, Google, May 10, 2023
- Bard becomes Gemini: Try Ultra 1.0 and a new mobile app today, Google, February 8, 2024
- Google's AI claims it is conscious and one engineer believes it, ScienceAlert, June 2022
- Google fires researcher who claimed LaMDA AI was sentient, Engadget, July 22, 2022 (reporting first published by Big Technology)