Emotion recognition software tries to infer feelings from faces, voices, or body signals, and the science under it is weaker than the marketing suggests. A 2019 review published in the journal Psychological Science in the Public Interest found that facial movements are not reliable or specific enough to diagnose an emotion, and that they vary across cultures and situations. In the European Union, the AI Act has prohibited inferring emotions in workplaces and education institutions since February 2, 2025, with exceptions for medical or safety reasons.
Key takeaways
- People do scowl, smile, and frown in line with common beliefs more often than chance, but not consistently enough to read a feeling from a face.
- A meta-analysis cited in the 2019 review found the expected facial configuration in only 19 percent of emotional episodes on average.
- Article 5(1)(f) of the EU AI Act forbids inferring emotions in the workplace or in education settings, with medical and safety exceptions.
- Counting smiles, detecting fatigue, and analyzing written text are treated by the Commission's guidelines as outside the ban.
- Since June 21, 2022, Microsoft has not offered emotion prediction in Azure Face to new customers.
What the 2019 review found
Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix Martinez, and Seth Pollak reviewed the evidence on whether emotions can be read from facial movements, and their paper appeared in Psychological Science in the Public Interest in 2019. They focused on six categories: anger, disgust, fear, happiness, sadness, and surprise. People do sometimes smile when happy or scowl when angry, more than chance would predict. Yet the review reports substantial variation across cultures, situations, and even within one situation, and the same facial configuration can express more than one emotion or something that is not an emotion at all.
Numbers from the review make the gap concrete. An expanded meta-analysis covered 89 effect sizes from 47 studies with 3,599 participants. The hypothesized facial configurations appeared in an average proportion of .19 of emotional events, with category values from .15 to .25, which the authors interpret as no evidence to weak evidence of reliability. In two field studies of winning judo fighters, only eight of 55 and 25 of 119 produced a Duchenne smile, and smiling tracked social interaction better than reported happiness.
Why software inherits the problem
A classifier that assigns the label "angry" to a face is only as valid as the link between that face and anger. The review notes that many studies showed participants only posed photographs, such as a scowling face standing in for anger, and ignored context. Software trained on such labels can be consistent without being correct. The review's authors also noted that technology companies were investing heavily in reading emotions from presumed facial expressions, and that several already claimed success.
The EU rule and what it covers
The AI Act's Article 5(1)(f) prohibits using AI systems to infer the emotions of a natural person in the areas of workplace and education institutions, except where the system is intended for medical or safety reasons. Recital 44 of the Act cites serious concerns about the scientific basis of such systems, naming limited reliability, lack of specificity, and limited generalizability. The Commission's guidelines on prohibited practices were published on February 4, 2025. They are non-binding, and the Court of Justice of the European Union has the final word on interpretation.
The guidelines work through examples. The table below summarizes several of them. It is a simplified orientation, not legal advice, and an organization's counsel decides how the rule applies to a specific tool.
| Use case | Treatment under Article 5(1)(f), per the guidelines |
|---|---|
| Call center uses webcams and voice analysis to track employees' anger | Prohibited |
| Software infers emotional tone of staff on hybrid video calls | Prohibited |
| System infers from voice or gestures that a student is furious and about to become violent | Prohibited |
| Call center tracks customers' anger or impatience by voice | Not prohibited by this article |
| Broadcaster counts how often presenters smile on camera | Not emotion recognition |
| System detects fatigue in pilots or drivers | Not emotion recognition, because fatigue is a physical state |
| Tool infers emotion from written text only | Outside the ban, because the definition requires biometric data |
Emotion recognition outside workplaces and schools is not simply allowed. The guidelines say such systems are classed as high-risk under Annex III, and Article 50(3) adds transparency duties. According to the Commission's AI Act timeline, updated on October 19, 2026, the high-risk rules for Annex III use cases were postponed to December 2, 2027 by the AI Omnibus.
How vendors have responded
On June 21, 2022, Microsoft said it would withdraw the Azure Face capabilities that infer emotional states. Its Azure blog post on facial recognition safeguards made emotion detection unavailable to new customers from that date, while current customers kept access until June 30, 2023. The stated reasons were privacy concerns, the absence of a shared definition of emotions, and difficulty generalizing inferences across use cases, regions, and demographics.
Questions to put to any emotion AI vendor
- What exactly is inferred: an emotion, an attitude, a physical state, or a visible expression?
- What labels trained the system, and who assigned them?
- Has accuracy been measured in the settings, regions, and demographic groups where it will be used?
- Does an output affect hiring, grading, pay, or discipline?
- Where would the tool fall under EU rules if used in a workplace or school?
This article covers the EU rule because it is the binding law found for this topic. No comparable ban was checked for Canada or the United States, so none is described here.
FAQ
Can AI reliably tell whether someone is angry or happy?
Not from facial movements alone. The 2019 review found that expected facial configurations occur more often than chance but too inconsistently to diagnose an emotion, and context changes what a face means.
Is emotion recognition banned in the EU?
Partly. Emotion inference is prohibited where it targets employees at work or learners in education institutions, with medical or safety exceptions. Other uses are not banned by that article but face high-risk and transparency rules.
Does detecting a smile count as emotion recognition?
According to the Commission's guidelines, observing that a person is smiling is not emotion recognition. Concluding from the smile that the person is happy is.
Does the EU ban cover emotion detection in text?
The guidelines say a system that infers emotions from written text is not based on biometric data and falls outside the prohibition.
Related reading
- What AI cannot do
- Limits of AI in classrooms
- Whether AI can feel emotions
- How pattern recognition works
More in AI Policy and Ethics.
Last verified: October 2026
Sources
- Barrett LF et al., Emotional Expressions Reconsidered (review article), Psychological Science in the Public Interest, 2019
- Regulation (EU) 2024/1689 (AI Act), Article 5(1)(f), Recital 44, Annex III point 1(c), Official Journal of the European Union, 2024
- Commission Guidelines on prohibited artificial intelligence practices, C(2025) 5052 final, European Commission, 2025
- AI Act regulatory framework and timeline page, European Commission, updated October 19, 2026
- Microsoft Azure blog, post on responsible AI work and facial recognition safeguards, 2022