Artificial intelligence is not one product, not a database of checked facts, and not automatically neutral. The OECD defines an AI system as a machine-based system that works out, from the inputs it is given, how to produce outputs, whether predictions, content, recommendations, or decisions. That definition covers a spam filter, a loan-scoring model, and a chatbot alike, and none of them is guaranteed to be correct, so the useful question about any AI output is how it can be checked.
Key takeaways
- AI is a broad category; machine learning is one part of it, and chatbots are one application.
- A fluent answer from a language model can be false, and OpenAI lists plausible-sounding but incorrect answers among ChatGPT's known limitations.
- Lawyers in Mata v. Avianca (decided June 22, 2023) were penalized USD 5,000 over a filing that cited cases invented by ChatGPT.
- NIST's 2022 report on AI bias says bias comes from systemic and human factors as well as data, and that zero risk of bias is not achievable.
- Conversational programs are older than most readers assume: ELIZA was described in 1966.
What the term covers, and what it does not
The OECD definition quoted above is the wording shown on the organization's AI policy site. Two parts matter here. "Infers" separates AI systems from software that only executes instructions written out step by step, and "predictions, content, recommendations, or decisions" shows how wide the output range is. The same definition notes that systems differ in how autonomous they are and how much they adapt after deployment.
Terms nest. NIST treats machine learning as computer programs that use data to learn patterns, and places it within the scope of AI. A chatbot is then one kind of application built on those methods. Treating "AI" as a synonym for "chatbot" hides the AI that runs out of sight, such as a model that ranks incoming mail as spam.
Five common claims, checked
The table pairs claims often heard about AI with what primary sources actually say. It is a reading aid, not a ranking of risks.
| Claim | What the sources say |
|---|---|
| AI and chatbots are the same thing | The OECD definition covers any system that infers outputs from inputs, and NIST places machine learning inside AI. Chatbots are one application. |
| A confident answer is a correct answer | OpenAI's launch post for ChatGPT states that the model can give answers that sound plausible yet are incorrect or nonsensical. |
| AI looks things up, so it cannot invent citations | The Mata v. Avianca sanctions order records a brief that cited cases which did not exist, supplied by ChatGPT. |
| Machines are objective, so AI cannot be biased | NIST SP 1270 identifies systemic, statistical, and human bias, and says zero risk of bias in an AI system is not possible. |
| Talking with computers is a recent idea | Weizenbaum described the ELIZA program in Communications of the ACM in January 1966. |
Fluent is not the same as verified
OpenAI's announcement of ChatGPT on November 30, 2022, is unusually direct about the problem. It says the model can give answers that sound plausible yet are incorrect or nonsensical, and explains that fixing this is hard because, during reinforcement learning, there is no source of truth. Teaching the model to be more cautious also makes it decline questions it could answer correctly. A language model produces text that fits the pattern of the conversation, and the pattern can be right or wrong.
A public record of the consequences exists in the sanctions order in Mata v. Avianca, where a brief rested on cases that the chatbot had fabricated and the filers did not verify. The lesson is about checking, not about avoiding tools.
Bias does not live only in the data
A common reply to bias worries is that a model is only as biased as its data, so cleaner data fixes it. NIST's Special Publication 1270, issued in March 2022, argues for a wider view. It names three categories of bias, systemic, statistical, and human, and says that current remedies focus on computational factors such as dataset representativeness and algorithm fairness, while human and institutional factors are overlooked. Decisions about which data to use, what model to build, and whether AI is needed at all carry the assumptions of the people and organizations making them.
The report adds that bias is neither new nor unique to AI and that achieving zero risk of bias in an AI system is not possible. The practical conclusion is to test for specific harms in the setting where the system will be used and to keep a person accountable for the outcome.
An older idea than it looks
ELIZA, described in 1966, worked by spotting key words in typed input and filling in templates, and its author wrote that people supply much of the sense of being understood. That is a different technique from today's language models, but the misconception it produced, that fluent replies mean comprehension, is the same one that surrounds modern chatbots. Knowing the history makes the question of what a system actually does easier to ask.
Five questions before trusting an AI output
- Can the claim be checked against an original source, such as a statute, a case, a dataset, or a primary study?
- Was the system built for this task, or is a general tool being stretched to fit?
- Who is accountable if the output is wrong, and is that person reviewing it?
- Has the system been tested on people and cases like the ones affected?
- Does the stake justify a second opinion from a qualified professional?
This page offers general information, not legal or professional advice. Where an AI output affects money, health, or legal rights, the decision belongs with a qualified person and the official sources.
FAQ
Is AI the same as machine learning?
No. Machine learning is a set of methods in which programs learn patterns from data, and NIST describes AI as including machine learning within its scope. Other approaches, such as hand-written rules, have also been called AI.
Can a chatbot make up sources?
Yes. In Mata v. Avianca, lawyers filed a brief whose cited cases came from ChatGPT and did not exist. Any citation from a chatbot should be located in an original database before it is relied on.
Can AI be unbiased?
NIST says it is not possible to achieve zero risk of bias in an AI system. Bias can be measured and reduced for a specific use, but it cannot be assumed away.
Is conversational AI new?
The technology behind current chatbots is recent, but the idea is not. Weizenbaum's ELIZA program, described in 1966, held typed conversations using keyword rules.
Related reading
- What counts as AI and what does not
- A beginner's AI learning path
- Canada's approach to AI regulation
- Computer vision as a form of AI
More in AI Basics.
Last verified: October 2026
Sources
- AI system definition, OECD AI Policy Observatory (OECD AI), OECD Recommendation on AI, as shown October 2026
- Schwartz R. et al., Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, NIST Special Publication 1270, National Institute of Standards and Technology, March 2022
- Introducing ChatGPT, OpenAI, November 30, 2022
- Sanctions order, Mata v. Avianca, Inc., docket 22-cv-1461, S.D.N.Y., June 22, 2023
- Weizenbaum J., the ELIZA paper, Communications of the ACM, Volume 9, Number 1, January 1966