Whether AI is inherently unethical is not something a single rule or study can settle, because the documented harms trace back to design choices, missing oversight, and the way systems were used. The best-documented public-sector example is the Dutch childcare benefits scandal: a fraud risk model introduced in 2013 used "Dutch citizenship: yes/no" as an input, the Dutch Data Protection Authority fined the tax administration EUR 2.75 million in December 2021, and the cabinet resigned on January 15, 2021. Each failure in that case was a decision that people made and could have made differently.
Key takeaways
- "Inherently unethical" would mean every AI system causes the harm regardless of design, and no source reviewed here shows that.
- Amnesty International found that the Dutch risk model gave higher scores to applicants without Dutch nationality and that no human rights assessment was done before use.
- The scandal also involved harsh rules and rigid readings of the law, so the algorithm was one part of a larger failure.
- Canada's Directive on Automated Decision-Making requires federal institutions to complete an Algorithmic Impact Assessment, but it does not cover private companies.
- A practical test asks what the system decides, what data it uses, who can challenge a result, and who checks it.
What the claim would have to mean
A technology is inherently unethical if the harm comes with it in every form and cannot be designed away. General-purpose software is different: the same statistical method can sort photos, forecast demand, or score people for fraud, and only the last of those can change a family's income. Ethical judgments therefore attach to a specific system in a specific setting.
That framing is testable. For any reported harm, the useful questions are whether the harm followed from the method itself, from the data, from a design choice, or from how institutions acted on the output. If the answer is a choice, the harm is contingent, and rules, audits, and redesign are the remedy. The Dutch case is useful because the records are detailed enough to separate those causes.
The Dutch childcare benefits case
The Netherlands paid childcare allowances that its tax administration also policed for fraud. Amnesty International's 2021 report "Xenophobic Machines" says that in 2013 the administration adopted an algorithmic "risk classification model" with self-learning elements to flag applications as likely to be inaccurate or fraudulent. Parents selected by it had benefits suspended and were investigated. Amnesty reports that "Dutch citizenship: yes/no" was a parameter, so applicants of other nationalities received higher risk scores, and it characterizes this as racial profiling.
A parliamentary inquiry committee published its report "Ongekend onrecht" (unprecedented injustice) on December 17, 2020. On January 15, 2021, Prime Minister Mark Rutte and his cabinet resigned; press coverage of the time put the number of families forced to repay tens of thousands of euros at roughly 10,000, while Amnesty describes tens of thousands of parents and caregivers falsely accused. Estimates differ with the definition used, and this article does not choose between them.
Which parts were choices
The table separates the elements of the case. It relies on Amnesty's report and the regulator's findings, and it is a simplified reading rather than a legal analysis.
| Element | What happened | Why it was a choice |
|---|---|---|
| Input data | Nationality was a model parameter | Designers decided which features to include; a model can be built without them |
| Self-learning | The model adjusted over time with little meaningful human oversight, which Amnesty calls a discriminatory loop | Organizations decide whether outputs are reviewed before they trigger action |
| Prior assessment | No human rights risk assessment before use | An assessment can be required in advance |
| Transparency | Affected parents could not learn why they were selected | Notice and explanation can be built into the process |
| Data retention | The regulator said dual-nationality records that should have been deleted in January 2014 were still held in 2018 | Deletion schedules are an organizational duty |
What the case does not show
The scandal is not evidence that all risk scoring is unlawful or that every machine learning model discriminates. Amnesty's report itself says the affair also involved harsh enforcement policies, rigid interpretations of the law, and an administrative sanction system, and the model was only one part of it. The Dutch Data Protection Authority's finding concerned the processing of nationality data, which it called unlawful and discriminatory, not the use of statistics as such.
The case also predates current rules. Whether a comparable system would face a different outcome today depends on the jurisdiction, the sector, and the rule in force, which is why the next section looks at one enacted Canadian rule rather than at general principles.
What an enacted Canadian rule requires
Canada's Algorithmic Impact Assessment is a required risk questionnaire that supports the Treasury Board's Directive on Automated Decision-Making. It contains 65 risk questions and 41 mitigation questions, and its score places a system in one of four impact levels, from minimal impact to very high impact. The level sets the safeguards the Directive requires. The Directive is a policy for federal institutions, so it governs how departments use automated decision systems, not how a private lender or employer does. Private-sector systems fall under other law, including privacy statutes, which the related article on Canada's AI regulation explains.
The design logic matches the Dutch lessons: rate the system's impact before use, scale the safeguards to that rating, and keep a record that others can inspect.
Questions that separate a harmful system from a defensible one
- What decision does the output influence, and how serious are the consequences if it is wrong?
- Which inputs does the system use, and could any of them stand in for nationality, ethnicity, religion, or other protected traits?
- Does a person review a flag before benefits are stopped or an investigation begins?
- Can an affected person learn why they were selected and challenge the result?
- Was an impact assessment completed before launch, and is it published?
- Who monitors results for group differences after deployment, and what happens when a gap appears?
This article gives general information, not legal advice. Whether a particular system complies with a law or policy depends on its facts and its jurisdiction, and questions about legal duties belong with a qualified lawyer or the regulator named in the sources below.
FAQ
Is any AI technology banned as inherently unethical?
Some uses are prohibited by specific laws, but those bans target practices, not the technology as a whole. The scope and status of each ban depend on the jurisdiction, so the text of the relevant law is the place to check.
Was the Dutch risk model machine learning?
Amnesty describes it as an algorithmic system with self-learning elements. Public descriptions do not give enough technical detail to classify the model more precisely, so this article does not go further.
Does Canada's Directive apply to banks or employers?
No. It is a Treasury Board policy for federal institutions. Private organizations are subject to other laws, such as privacy and human rights statutes.
What did the regulator actually penalize?
The Dutch Data Protection Authority fined the tax administration for processing applicants' dual nationality in an unlawful and discriminatory way and for keeping data that should have been deleted. The fine was announced in December 2021.
Related reading
- How Canadian law treats AI today
- The ethical dilemma of building moral machines
- Arguments against AI as a social threat
- Threat or tool: where the debate stands
More in AI Policy and Ethics.
Last verified: October 2026
Sources
- Xenophobic Machines: Discrimination Through Unregulated Use of Algorithms in the Dutch Childcare Benefits Scandal, Amnesty International, October 25, 2021
- Algorithmic Impact Assessment, Treasury Board of Canada Secretariat, 2026
- Ongekend onrecht (final report), Parliamentary Inquiry Committee on Childcare Benefits, Dutch House of Representatives, December 17, 2020
- Dutch PM Rutte and his government quit over child welfare scandal, Al Jazeera, January 15, 2021
- Tax office fined EUR 2.7m for discriminating against dual national families, DutchNews.nl, December 2021